WhatsTheBigData

[slideshare id=45132578&doc=pythondatascienceatzillow-150225104833-conversion-gate02]

We just started to use the term “data scientist” and the demise of this new profession is already predicted? Well, at least it’s not one more “rise of the machines” prophecy; it’s the provocative title of a proposed panel for the upcoming SXSW.

The organizer of the panel, Scott Hendrickson of Gnip, has provided a useful run-down of some of the arguments for and against the possible disappearance of data scientists. Supporting the proposition are the current scarcity of data science talent and a slew of startups providing “data science as a service.” As an example of the opposition to the “democratization of algorithms,” Hendrickson quotes Cathy (Mathbabe) O’Neil who wrote recently that “if your model fails, you want to be able to figure out why it failed. The only way to do that is to know how it works to begin with. Even if it worked in a given situation, when you train on slightly different data you might run into something that throws it for a loop, and you’d better be able to figure out what that is.” In other words, machines will never have the deep understanding of the tools of data science that is required to practice data science.   Continue reading →

Source: A History of the International Federation of Classification Societies

As ones and zeros eat the world, data is the new product and data science is the new process of innovation.

The International Institute for Analytics predicts that in 2014 companies in a variety of industries will increasingly use analytics on the data they have accumulated to develop new products and services. NewVantage Partners’ most recent Big Data Survey reports that 68% of executives felt that “new product innovations” was the greatest value to their organization from big data. In releasing the Accenture Technology Vision 2014, Accenture’s CTO Paul Daugherty said that “Digital is rapidly becoming part of the fabric of [large enterprises’] operating DNA and they are poised to become the digital power brokers of tomorrow.”

The best example of this trend I’ve encountered recently came from an industry one does not necessarily associate with data crunching and analysis—the vehicle remarketing industry, better known as used cars auctions. In 2012, Manheim, a subsidiary of Cox Enterprises, handled nearly 8 million used vehicles, facilitating transactions representing more than $50 billion in value.  With annual revenues of more than $2.5 billion, Manheim offers its services in 14 countries, from physical and online auction channels to financing, transportation, and mobile solutions. Manheim’s research and consulting arm, Manheim Consulting, provides market intelligence and publishes the monthly Used Vehicle Value Index and the annual Used Car Market Report (see here for the 2014 version).

Manheim has provided for free this type of analysis, seeing it as part of the value it offers to auto dealers who are members of its network.  But now it has moved into using its deep knowledge of the used car market and its analytics expertise to offer a new, fee-based service.  Shifting the analytics team from supporting the business to generating revenues, “we’ve decided to look at how we can help dealers in managing the risk associated with their inventory,” T. Glenn Bailey told me.

Bailey is Senior Director of Enterprise Product Planning at Manheim, and his responsibilities include market segmentation, forecasting, and optimization.  He and his team started testing last year a new service called DealShield. The idea came from the financial markets, specifically put option contracts. Just like a put option protects the buyer from a decline in the price of a stock below a specified price, so does DealShield offer a guarantee that Manheim will buy a car back from the dealer, within a certain time frame, for what they paid for it plus the fee they paid.  “It is as if they never bought the car,” says Bailey.

Manheim’s market knowledge and analytics skills give it confidence in its estimates of the value of a car and what they would be able to offer for it if it comes back to them. “We see a lot of value in it,” says Bailey, “because one of the things dealers like to have is liquidity. They use wholesale financing to buy used cars and typically repay the loan within seven to fourteen days. The inventory that’s sitting out there is money that is tied up. DealShield allows them to get out of that car and get their money back in a certain period of time.”

To do their analysis, the Manheim team uses tools that have served this purpose for years, demonstrating that for certain types of analysis and data you can do data science without using any of the new big data technologies. The data is collected and stored in an IBM DB2 database and the analysis is done using a variety of SAS analytics tools.  “The need to combine data from different sources is why we moved into a SAS cloud,” says Bailey. “I wanted our analyst team to be focused on the analytics and not worry about the administrative side.”

Speaking of the analyst team, Bailey says that “we are in the same market for analytics and data science talent with everybody.” In the competition for these hard-to-find professionals, Bailey looks for creativity, communications skills and willingness to learn the business. “In my experience,” he says, “it is fairly easy to tell if you have the technical chops.” He spends most of the time when he interviews people trying to determine if they are creative and can come up with new ideas on how to apply analytics tools to the data to find new insights. “Reversing the flow of cause and effect,” Bailey calls it. “Maybe optimization can tell us where to send a vehicle to maximize value.”

In addition to looking for “people that can bring technology to the business,” Bailey also looks for people who are comfortable with “getting with the business itself.” He calls it “putting on the polo shirt,” spending time with the dealers and getting engaged with them to understand their business first-hand.  This practical bent does not stop with the hiring of the right people but continues with establishing the right work environment and a “fail-fast” culture. “In some sense,” says Bailey, ”failure is rewarded because it means you are testing this thing out.” When they developed DealShield, “we had a chart that over a 2-month period showed all the things that failed. If it doesn’t work, kill it.”

In addition to being the first knowledge-based service that is expected to bring in a new revenue stream, DealShield breaks new ground for Manheim because it is the first time the company actually owns cars (when they come back from the dealer), not just acting as a middle-man. That became an opportunity for an analyst on Bailey’s team to hone further her knowledge of the business.  “She is now responsible for selling the cars. She is setting the auction, the floor price, where to run the auction,” says Bailey.

Doing data science means engaging with the business, inventing new data-based products, even becoming an integral part of revenue stream for the business.

[Originally published on Forbes.com]

Data scientists are data junkies—when they see a new data set they are just naturally excited and can’t wait to explore.

Mingsheng Hong is Chief Data Scientist at Hadapt, a Boston-based startup that offers an analytical platform that integrates structured and unstructured data in one cloud-optimized system. Before joining Hadapt, Mingsheng was Field CTO for Vertica. He holds a Ph.D. in Computer Science from Cornell and a BSc in Computer Science from Fudan University. Mingsheng is president of NECINA and is active in St. Baldrick’s Foundation, a volunteer-driven charity that funds research to find cures for childhood cancers. I talked to Mingsheng just before he shaved his head, a visual indicator and act of solidarity expected from successful St. Baldrick’s fundraisers.

As a graduate student, were you thinking of an academic career?

At Cornell, I explored both academic and private industry career tracks. I love research and innovation, and discovered my passion for explaining ideas to people from various backgrounds and getting them excited about these ideas. While that aligns with a more academic track, in the end I decided the private sector was a better fit for me. I’m driven by the challenge of taking an idea and carrying it end-to-end, from idea to product development to sales. During graduate school, I had the opportunity to visit Microsoft for a few summers, and I got a lot of exposure to database R&D and came away with a good feel for the industry. My research work there was commercialized in SQL Server 2008 and 2012, which was very exciting.   Continue reading →