Survey: The Hunt for Unicorn Data Scientists Boosts the Salaries of Predictive Analytics Professionals

Burtch1

Base Salaries for Individual Contributors

Burtch2

Base Salaries for Managers

Unicorn Data Scientists (upgraded from “sexy data scientists”) are hard to find and are paid more than $200,000 per year. A new survey finds that the rising data science tide lifts the compensation of all other data analytics professionals, even if they don’t know how to code.

The Burtch Works Study: Salaries for Predictive Analytics Professionals is based on interviews with 1,757 data analytics professionals conducted over the 12 months ending April 2015 by executive recruiting firm Burtch Works. It is a unique source of information in that it does not rely on self-reporting or data provided by human resources departments. It also provides insights into how the demand for data scientists impact the salaries of other data analytics professionals because it excludes data scientists, covered in a separate Burtch Works study, published earlier this year (I wrote about that study here).

Burtch Works defines predictive analytics professionals as those who can “apply sophisticated quantitative skills to data describing transactions, interactions, or other behaviors of people to derive insights and prescribe actions.” Data scientists are a subset of this group—they have the “computer science skills necessary to acquire and clean or transform unstructured or continuously streaming data, regardless of its format, size, or source.”

The additional computer science skills put data scientists on top in terms of compensation regardless of their levels of experience and managerial responsibilities but predictive analytics professionals are keeping up, seeing their salaries and bonuses rise. For example, the median base salary for the most experienced individual contributors rose from $115,250 last year to $125,000 this year and for managers managing teams of ten or more the median base salary rose from $225,000 to $235,000.

Predictive analytics professionals continue to benefit from the increasing demand and short supply for their quantitative analysis skills. The median base salary of individual contributors varies from $76,000 for those at level 1 (0 to 3 years of experience) to $125,000 for those at level 3 (9+ years of experience). The median bonus received varies from $8,100 to $18,100, depending on job level.

The median base salary of managers varies from $125,500 for those at level 1 (1 to 3 reports) to $235,000 for those at level 3 (10+ reports). The median bonus received by managers varies from $23,000 to $75,000 depending on job level.

More and more people are attracted by the demand for data analytics professionals and the potential to become a unicorn. Data recently released by the National Center for Education Statistics, according to Phys.org, shows bachelor’s degrees in statistics grew 17% from 2013 to 2014. This marks 15 consecutive years the number of undergraduates in statistics has risen, increasing by more than 300% since the 1990s. In addition, from 2000 to 2014, master’s and doctorate degrees in statistics also grew significantly at 260% and 132%, respectively.

“The Bureau of Labor Statistics projects job growth for statisticians will increase 27% between 2012 and 2022, outpacing the projected 11% rate for all other occupations. The number of graduates in statistics each year—approximately 2,000 bachelor’s degrees, 3,000 master’s degrees and 575 doctorate degrees—seems unlikely to match this demand,” says Phys.org.

Originally published on Forbes.com

Posted in Data Science, Predictive analytics | Tagged , , | Leave a comment

How to Become a Unicorn Data Scientist and Make More than $240,000

What makes a good data scientist? And if you are a good data scientist, how much should you expect to get paid?

Owen Zhang, ranked #1 on Kaggle, the online stadium for data science competitions, lists his skills on his Kaggle profile as “excessive effort,” “luck,” and “other people’s code.” An engineer by training, Zhang says in this ODSC interview that data science is finding “practical solutions to not very well-defined problems,” similar to engineering. He believes that good data scientists, “otherwise known as unicorn data scientists,” have three types of expertise. Since data science deals with practical problems, the first one is being familiar with a specific domain and knowing how to solve a problem in that domain. The second is the ability to distinguish signal from noise, or understanding statistics. The third skill is software engineering.

Not having formal education in statistics or software engineering, Zhang explains that he acquired his data science skills by competing in Kaggle and learning from its community. No doubt being very good at learning on your own is a required skill, to say nothing about hanging out with the right people, preferably unicorn data scientists. Galit Shmueli, Professor of Business Analytics at NTHU, told rjmetrics that her one piece of advice for data scientists just getting started is to “attend a conference or two, see what people are working on, what are the challenges, and what’s the atmosphere.”

Recent data shows that unicorn data scientists can make more than $240,000 annually. This according to the 2015 Data Science Salary Survey where O’Reilly Media’s John King and Roger Magoulas report the results of a survey of 600 “data practitioners” (reflecting the recency of the term, only one-quarter of the respondents have job titles that explicitly identify them as “data scientists”).

The median annual base salary of the survey sample is $91,000, and among U.S. respondents is $104,000, similar to last year’s results. 23% said that it would be “very easy” for them to find another position.

Keep in mind that “23% of the sample hold a doctorate degree,” and additional 44% hold a master’s. The word “sample” here means, as it does in almost all other surveys today, “the people that wanted to answer our survey.” But unlike other survey report authors, King and Magoulas make sure to issue this warning: “We should be careful when making conclusions about survey data from a self-selecting sample—it is a major assumption to claim it is an unbiased representation of all data scientists and engineers… the O’Reilly audience tends to use more newer, open source tools, and underrepresents non-tech industries such as insurance and energy.”

Still, we can learn quite a lot about the background and skills required for admission into this well-paid group of data masters. Two-thirds of respondents had academic backgrounds in computer science, mathematics, statistics, or physics.

Beyond the initial training, it is important to keep abreast of the ever-changing landscape of data science tools: “It seems likely that in the long run knowing the highest paying tools will increase your chances of joining the ranks of the highest paid,” say King and Magoulas. And the most recent additions to the data science tool pantheon provide the greatest boost to salaries: “…learning Spark could apparently have more of an impact on salary than getting a PhD. Scala is another bonus: those who use both are expected to earn over $15,000 more than an otherwise equivalent data professional.”

The bad news is that the more time spent in meetings (even for non-managers), the more money a data scientist makes. Another widely discussed unpleasant part of the job—data cleaning—is the #2 task on which data scientists spend the most time, with 39% of survey participants spending at least one hour per day on this task. The good news is that exploratory data analysis is what occupies them most, with 46% spending one to three hours per day on this task and 12% spending four hours or more.

More data on the skills employed by practicing data scientists comes from an AnalyticsWeek survey of 410 data professionals. In Optimizing Your Data Science Team, Bob E. Hayes reports that respondents were asked to indicate their level of proficiency for 25 different skills.” Solving problems with data,” says Hayes, “requires expertise across different skill areas: 1) Business, 2) Technology, 3) Programming, 4) Math & Modeling and 5) Statistics. Proficiency in each skill area is related to job role.”

All of these skills may not present themselves in a single data scientist but it’s possible to assemble all of them by putting together a top-notch data science team. In “Tips for building a data science capability” from consulting firm Booz Allen Hamilton, we learn that “rather than illuminate a single data science rock star, it is important to highlight a diversity of talent at all levels to help others self-identify with the capability. It is also a more realistic version of the truth. Very rarely will you find ‘magical unicorns’ that embody the full breadth of math and computer science skills along with the requisite domain knowledge. More often, you will build diverse teams that when combined provide you with the ‘triple-threat’ (computer science, math/statistics, and domain expertise) model needed for the toughest data science problems.”

The concept of a data science team, combining various skills and educational backgrounds, is high on the agenda of the 175-year-old American Statistical Association (ASA) which is probably looking in dismay at the oodles of funds going to establishing new data science programs and research centers at American universities, to say nothing about the salaries of data scientists as opposed to the salaries of statisticians.

The ASA issued a “policy statement” on October 1, reminding the world that statistics is one of the three disciplines “foundational to data science” (the other two being database management and distributed and parallel systems, providing a “computational infrastructure”). The statement concludes with “The next generation [of statisticians] must include more researchers with skills that cross the traditional boundaries of statistics, databases and distributed systems; there will be an ever-increasing demand for such ‘multi-lingual’ experts.”

In other words, if you aspire to a $200,000+ salary, better call yourself a data scientist and start coding.

Posted in Data Science | Tagged , , | Leave a comment

Video: What is the Internet of Things (IoT)?

[youtube https://www.youtube.com/watch?v=07MZVjjuLKw?rel=0]

CTIA-The Wireless Association explains the opportunities the IoT will provide and what is needed in order to meet user demands (hint: spectrum!).

Posted in Internet of Things | Tagged | Leave a comment

Dell-EMC deal largest-ever tech acquisition (Infographic)

Infographic: Dell-EMC Deal Dwarfs Other Tech Acquisitions | Statista
You will find more statistics at Statista

Posted in Misc | Tagged , , | Leave a comment

Compatibility Research Inc., granddaddy of online dating (Video)

Video: http://espn.go.com/video/clip?id=espn:13415984

 Look Magazine, February 1966

Look Magazine, February 1966

Five thirty Eight:

In our modern age of Tinder, OkCupid and Match.com, we’re used to the idea that algorithms can help us find love. But while the algorithms may have improved as the market for online dating has expanded, the inputs — the questions these computer matchmakers ask dating hopefuls — haven’t changed much since the 1960s, when Compatibility Research Inc. launched the first computerized dating service.

OnlineDatingCompatabilityresearch

Onlinedating operation match 1965

onlinedating_operationMatch_quex

See also the Harvard Crimson article from November 3, 1965, about Compatibility Research’s Operation Match, which includes this ditty:

Well, I filled out my form and I sent it along,

Never hoping I’d get anything like this.

But now when I see her,

Whenever I see her,

I want to give her one great big I.B.M. kiss.

She’s my I.B.M. baby, the ideal lady,

She’s my I.B.M. baby.

From the first time I met her I couldn’t forget her,

She’s my I.B.M. baby.

Well we’ve dated sometime,

Things are going just fine, and I’d like to settle down with her.

Just like birds of a feather

We put 2 and 2 together, and we came one with an I.B.M. affair.

She’s my I.B.M. baby, I don’t mean maybe,

She’s my I.B.M. baby.

Today, one in 10 adults now spends, on average, an hour a day on a dating web site or app, according to Nielsen. Online dating in the U.S. was a $2.2 billion industry last year.

online-dating

Posted in Misc | Tagged , , | Leave a comment

IBM Watson and Ken Jennings Compete Again, This Time for Title of Most Productive (Video)

[youtube https://www.youtube.com/watch?v=lszB8muRqQA?rel=0]

Posted in Artificial Intelligence | Tagged , | Leave a comment

The Road to Zillions of Connected Things (IoT)

IoT_IDC

Fresh out of topping Gartner’s most hyped technologies list for the second year in a row, the Internet of Things (IoT) has kept its buzz going over the last couple of weeks with a series of announcements and new market analysis reports. First, here’s a sample of recent announcements:

  • September 20: Dialog Semiconductor has agreed to acquire Atmel Corporation for approximately $4.6 billion, combining forces in the mobile power, IoT and automotive markets, and addressing a “market opportunity of approximately $20 billion by 2019.”
  • September 18: Orange announced it is building a Low Power Wide Area (LPWA) network covering the whole of France, in line with its “ambition to become the number one operator for the Internet of Things.”
  • September 17: Alcatel-Lucent announced the acquisition of Mformation to provide service providers and enterprises with a secure, scalable, application-independent IoT security and control platform for use across multiple industries.
  • September 16: HCL Technologies announced it will jointly develop with IBM Internet of Things solutions and that the two companies will set up for that purpose an incubation center in Noida, India.
  • September 15: Salesforce has entered the Internet of Things market with its IoT Cloud.  Marc Benioff, Salesforce CEO, told the attendees of its Dreamforce annual conference: “With the Internet of Things, I’m more connected than ever. It’s truly a customer revolution.”
  • September 14: GE announced the creation of GE Digital, a new business unit led by Chief Digital officer (CDO) Bill Ruh, with the mission to “win in the Industrial Internet” (GE’s term for the Internet of Things).
  • September 14: IBM also announced a new business unit dedicated to conquering the Internet of Things market, led by new-hire Harriet Green.

We also learned more this month about the state-of-the-market for IoT and its potential impact from a number of new reports:

IDC (also here) shared the results of its survey of 2,350 IT and business decision makers in (mostly) large and medium-size enterprises worldwide:

  • Enterprise decision makers see the IoT as “strategic” (58%, especially in the health, transportation, and manufacturing industries) or “transformative” (24%, especially in IT and professional services); 13% are still ”considering” it (especially in government and financial services) and 4.1% think it’s “not important.”
  • IoT momentum is real and quantifiable:  17% of participants in the survey have already deployed IoT and 31% plan to do so this year. Less than 5% “have considered but decided against it.”
  • IoT strategies are global in scope, with enterprises in Asia/Pacific leading other regions with almost 55% of survey participants (compared, for example, with about 45% in the North America).
  • B2B is considered by survey participants as the place IoT will grow, a reversal from last year where a majority of survey participants thought the consumer IoT is where the action will be.
  • Shift in the location of processing the data: More survey participants this year will process the data generated by IoT sensors at the ”edge” rather than in the data center, a reversal  from last year’s survey.
  • Top drivers for creating an IoT strategy:  Increased productivity (14.2%), time to market (11.8%), and process automation (10.1%).
  • Top challenges for the IoT: Security, upfront costs, ongoing costs.
  • IoT is anyone’s game in 2016: Hardware and networking vendors have lost ground in their perception as “leaders,” while software vendors, analytics vendors, and device/component vendors have gained in market awareness/perception.  “Industrial Internet companies,” while not a category that was asked about last year, is at less than 10%–GE and other companies using this term have their work cut out for them to make it synonymous with IoT.

Gartner reiterated its forecast of more than 30 billion installed IoT units and estimated it will result in a 20% increase in potential revenue generated from software for manufacturers running ‘intelligent devices’.  The Internet of Things (IoT), in Gartner’s view, turns every manufacturer into a software provider, a transformation which will have profound impact on application strategy, architecture, development and integration.

Gartner recommends that manufacturers differentiate with software, increase the intelligence in their devices by adding software, and ensure they have the licensing and entitlements tools to manage the software.

Accenture estimates that based on current policy and investment trends, the IoT could add about $500 billion to China’s cumulative GDP by 2030. This would result in China’s GDP being 0.3 percent higher in that year compared with current projections. However, by taking additional measures to improve its capacity to absorb IoT technologies and increase IoT investment, China could boost its annual GDP by 1.3 percent by 2030, cumulatively adding $1.8 trillion to the economy by that time.

Last but not least, Harvard Business School professor Michael Porter and PTC CEO Jim Heppelmann published in the Harvard Business Review “How Smart, Connected Products Are Transforming Companies.” They describe the impact of the IoT on the organizational structure of manufacturing companies and conclude that “Smart, connected products reshape not only competition, as we detailed in our previous article, but the very nature of the manufacturing firm, its work, and how it is organized. They are creating the first true discontinuity in the organization of manufacturing firms in modern business history.” They also see broader benefits of the IoT, including changing consumption patterns: “Smart, connected products will free us to purchase only the goods and services we need, to share products that we do not use much, and to get more out of the products that we already have. Instead of tossing out old products for the next generation, we will hold on to products that are continually improved, upgraded, and modernized.”

Originally published on Forbes.com

Posted in Misc | Leave a comment

What Makes a Good Data Scientist?

Data_Scientist_infographic_WR

Posted in Data Science | Tagged | Leave a comment

Creative Destruction and the ‘Uber Effect’

CBInsights_uber-vs-TAXI-valuation-chart

CB Insights:

We used CB Insights’ valuation data to look at how the rise of Uber’s valuation correlates with the market capitalization of Medallion Financial Corp (NASDAQ: TAXI). Medallion Financial is a publicly-traded company that originates, acquires, and services loans used to purchase taxi medallions in several large US urban markets that Uber is also active in, including New York. We charted the stock price of TAXI versus the valuations for many of Uber’s rounds since 2010.

We found that TAXI has also been hammered by an “Uber Effect,” with its price down even more than the decline seen by New York City medallions. TAXI’s stock price is down nearly 49% since Uber raised its breakout $258M Series C at a $3.5B valuation. (The NASDAQ is up ~26% in the same time period.)

Uber’s valuation is up over 13x.

Mark J. Perry at the American Enterprise Institute:

UberEffect1

UberEffect2

In 1942, economist Joseph Schumpeter described “creative destruction” as a “process of industrial mutation that incessantly revolutionizes the economic structure from within, incessantly destroying the old one, incessantly creating a new one.” There probably hasn’t been a better example of Schumpeterian creative destruction in the last decade or more than the recent ascendance of app-based ride-sharing services like Uber (and Lyft, Sidecar, Gett, Via, etc.)  challenging traditional, legacy taxi cartels in cities like New York, San Francisco, Chicago and more than 160 other US cities. Market-based evidence of the gale of creative destruction in the transportation industry is displayed in the two charts above. The top chart above shows how the increasing popularity of ride-sharing apps like Uber has caused the price of New York City individual taxi medallions to collapse by at least 37%, from a peak of more than $1 million in August 2013 to only about $650,000 in recent months (based on advertised asking prices here, not actual sales).

Further evidence of the “Uber effect” is displayed in the bottom chart above, showing the collapse in the stock price of Medallion Financial Corporation, from $16.45 in November 2013 to below $7 per share in the last few days. Medallion Financial Corporation (NASDAQ: TAXI) is a NYC-based specialty finance company that originates, acquires, and services loans that finance taxicab medallions. Just as the sky-high taxi medallion prices have been significantly eroded due to competition from the upstart ride-sharing services, so has the value of Medallion Financial Corporation’s stock price been significantly dropping. After tracking the SP&500 Index closely for many decades, the share price of Medallion Financial has fallen by a whopping 58% from its November 2013 peak, during a time when the S&P 500 has increased by 7.1%.

As the traditional, legacy taxi industry continues to collapse under the Schumpeterian forces of market disruption, the taxi cartels like the one in NYC are asking for taxpayer bailouts, or at least taxpayer-supported guarantees for taxi medallion loans. Consumers are the obvious winners from the creative destruction in the transportation industry – we now have more choice, better and faster service, friendlier drivers, cleaner cars, and maybe most importantly — lower prices. Traditional taxi drivers and medallion owners, after being protected from competition by government regulations for many generations, are the obvious losers from the “Uber effect.” Medallion prices will continue to fall as the taxi cartels continue to crumble and collapse.

NPR Planet Money: Listen to Episode 643, July 31, 2015, on Gene Freidman, the “Taxi King” and how his empire is starting to crumble. Also, “Why Does A Taxi Medallion Cost $1 Million?” from 2011.

Posted in Misc | Tagged , | Leave a comment

10 Predictions for Digital and IT Transformation: Gartner

Gartner-crystalball

Gartner released today its top predictions for “the digital future… an algorithmic and smart machine-driven world where people and machines must define harmonious relationships”:

1)    By 2018, 20 percent of business content will be authored by machines.
Technologies with the ability to proactively assemble and deliver information through automated composition engines are fostering a movement from human- to machine-generated business content. Data-based and analytical information can be turned into natural language writing using these emerging tools. Business content, such as shareholder reports, legal documents, market reports, press releases, articles and white papers, are all candidates for automated writing tools.

2)    By 2018, six billion connected things will be requesting support.
In the era of digital business, when physical and digital lines are increasingly blurred, enterprises will need to begin viewing things as customers of services — and to treat them accordingly. Mechanisms will need to be developed for responding to significantly larger numbers of support requests communicated directly by things. Strategies will also need to be developed for responding to them that are distinctly different from traditional human-customer communication and problem-solving. Responding to service requests from things will spawn entire service industries, and innovative solutions will emerge to improve the efficiency of many types of enterprise.

3)    By 2020, autonomous software agents outside of human control will participate in five percent of all economic transactions.
Algorithmically driven agents are already participating in our economy. However, while these agents are automated, they are not fully autonomous, because they are directly tethered to a robust collection of mechanisms controlled by humans — in the domains of our corporate, legal, economic and fiduciary systems. New autonomous software agents will hold value themselves, and function as the fundamental underpinning of a new economic paradigm that Gartner calls the programmable economy. The programmable economy has potential for great disruption to the existing financial services industry. We will see algorithms, often developed in a transparent, open-source fashion and set free on the blockchain, capable of banking, insurance, markets, exchanges, crowdfunding — and virtually all other types of financial instruments

4)    By 2018, more than 3 million workers globally will be supervised by a “robo-boss.”
Robo-bosses will increasingly make decisions that previously could only have been made by human managers. Supervisory duties are increasingly shifting into monitoring worker accomplishment through measurements of performance that are directly tied to output and customer evaluation. Such measurements can be consumed more effectively and swiftly by smart machine managers tuned to learn based on staffing decisions and management incentives.

5)    By year-end 2018, 20 percent of smart buildings will have suffered from digital vandalism.
Inadequate perimeter security will increasingly result in smart buildings being vulnerable to attack. With exploits ranging from defacing digital signage to plunging whole buildings into prolonged darkness, digital vandalism is a nuisance, rather than a threat. There are, nonetheless, economic, health and safety, and security consequences. The severity of these consequences depend on the target. Smart building components cannot be considered independently, but must be viewed as part of the larger organizational security process. Products must be built to offer acceptable levels of protection and hooks for integration into security monitoring and management systems.

6)    By 2018, 45 percent of the fastest-growing companies will have fewer employees than instances of smart machines.
Gartner believes the initial group of companies that will leverage smart machine technologies most rapidly and effectively will be startups and other newer companies. The speed, cost savings, productivity improvements and ability to scale of smart technology for specific tasks offer dramatic advantages over the recruiting, hiring, training and growth demands of human labor. Some possible examples are a fully automated supermarket or a security firm offering drone-only surveillance services. The “old guard” (existing) companies, with large amounts of legacy technologies and processes, will not necessarily be the first movers, but the savvier companies among them will be fast followers, as they will recognize the need for competitive parity for either speed or cost.

7)    By year-end 2018, customer digital assistant will recognize individuals by face and voice across channels and partners.
The last mile for multichannel and exceptional customer experiences will be seamless two-way engagement with customers and will mimic human conversations, with both listening and speaking, a sense of history, in-the-moment context, timing and tone, and the ability to respond, add to and continue with a thought or purpose at multiple occasions and places over time. Although facial and voice recognition technologies have been largely disparate across multiple channels, customers are willing to adopt these technologies and techniques to help them sift through increasing large amounts of information, choice and purchasing decisions. This signals an emerging demand for enterprises to deploy customer digital assistants to orchestrate these techniques and to help “glue” continual company and customer conversations.

8)    By 2018, two million employees will be required to wear health and fitness tracking devices as a condition of employment.
The health and fitness of people employed in jobs that can be dangerous or physically demanding will increasingly be tracked by employers via wearable devices. Emergency responders, such as police officers, firefighters and paramedics, will likely comprise the largest group of employees required to monitor their health or fitness with wearables. The primary reason for wearing them is for their own safety. Their heart rates and respiration, and potentially their stress levels, could be remotely monitored and help could be sent immediately if needed. In addition to emergency responders, a portion of employees in other critical roles will be required to wear health and fitness monitors, including professional athletes, political leaders, airline pilots, industrial workers and remote field workers.

9)    By 2020, smart agents will facilitate 40 percent of mobile interactions, and the postapp era will begin to dominate.
Smart agent technologies, in the form of virtual personal assistants (VPAs) and other agents, will monitor user content and behavior in conjunction with cloud-hosted neural networks to build and maintain data models from which the technology will draw inferences about people, content and contexts. Based on these information-gathering and model-building efforts, VPAs can predict users’ needs, build trust and ultimately act autonomously on the user’s behalf.

10) Through 2020, 95 percent of cloud security failures will be the customer’s fault
Security concerns remain the most common reason for avoiding the use of public cloud services. However, only a small percentage of the security incidents impacting enterprises using the cloud have been due to vulnerabilities that were the provider’s fault. This does not mean that organizations should assume that using a cloud means that whatever they do within that cloud will necessarily be secure. The characteristics of the parts of the cloud stack under customer control can make cloud computing a highly efficient way for naive users to leverage poor practices, which can easily result in widespread security or compliance failures. The growing recognition of the enterprise’s responsibility for the appropriate use of the public cloud is reflected in the growing market for cloud control tools. By 2018, 50 percent of enterprises with more than 1,000 users will use cloud access security broker products to monitor and manage their use of SaaS and other forms of public cloud, reflecting the growing recognition that although clouds are usually secure, the secure use of public clouds requires explicit effort on the part of the cloud customer.

Posted in Misc | Tagged , | Leave a comment