In its most recent hype cycle for emerging technologies, Gartner introduced “citizen data science” and “advanced analytics with self-service delivery.” Both technologies were predicted to reach the “plateau of productivity” in 2 to 5 years before
The shortage of data scientists and the resulting high salaries they command is giving rise to new self-service tools, automating all stages of data science so business analysts, marketing managers, IT staff and others could perform advanced analytics as part of their jobs.
By 2017, Gartner says, the number of these citizen data scientists in small and large organizations will grow five times faster than the number of highly skilled data scientists. Forrester agrees that the “huge demand” for data scientists will not be met in the short term, “even as more degree programs launch globally.” And the demand for advanced data analysis will only increase in the coming years with the rise of the Internet of Things.
Automation also helps the few overworked data scientists available today, making the experienced more productive and helping the newly-minted add value faster. A number of startups, such as Trifacta and Tamr, have focused on the early stages of the data analytics process—data preparation and transformation—and others have focused on later stages such as data visualization or on specific applications and industries.
An interesting challenge is automating the core of the data science process, the development and maintenance of predictive models (Forrester recently declared that Predictive Analytics is the hottest big data technology). The founders of DMway, which recently raised $1 million dollars in seed funding from JVP Labs, have “spent their entire careers on understanding and mapping the methods of algorithm and model developers,” says CEO Gil Nizri.
“Predictive analytics is a great competitive differentiator but it is still beyond the reach of most organizations,” adds Nizri. “DMway is enabling any size company, from SMB to enterprise, to compete on a level playing field.”
DMway’s model building “mimics the way a human expert develops a model,” says CTO Ronen Meiri. It starts by exploring the data, searching through all potential predictors and selecting the most influential. Using the set of influential predictors it creates a final prediction model and then applies it to an independent dataset to check its accuracy and over-fitting, making sure the model is general enough to apply to new observations. Finally, it provides multiple methods for seamless integration and deployment of the model.
The result is faster model development and more accurate models, sometimes 20% more accurate than traditionally-developed models. The benefits of automation, however, do not apply only to the initial development of the model. “Most of the resources are going to model maintenance and not to building the model for the first time,” says Meiri. “In micro-financing, for example, they usually re-build the model every three months.”
Businesses operating in environments with fast-changing conditions are prime candidates for automated model maintenance and a number of DMway’s early customers are Fintech startups. BACKED, providing loans to young Americans, uses DMway to predict loan defaults and Fido Credit, provider of micro-financing in Africa, uses DMway to assess credit risk. Beyond the financial sector, DMway’s automated model development is used by the marketing department of YES, a Cable TV operator, to predict customer churn and facilitate lead conversion.
As Eric Siegel, founder of Predictive Analytics World and author of Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, declaimed in a previous version of this rap video:
[youtube https://www.youtube.com/watch?v=bSP3z0LmWEg?rel=0]
Modeling means modifying models incrementally,
With a geek technique to tweak, it will reach the peak eventually.
Each step is taken to improve prediction on the training cases,
One small step for man; one giant leap—the human race is going places!
DMway is a good example of how automation is best discussed as human augmentation rather than human replacement, as it facilitates analyst-machine collaboration. The human race may indeed go places when data scientists—both of the highly skilled and of the “citizen” varieties—are supplied with tools that increase their productivity and the accuracy of models that drive decisions.
Originally published on Forbes.com
