Why Ones and Zeros Are Eating the World

30 years ago today, Steve Jobs unveiled the Macintosh. More accurately, The Great Magician took it out of a bag and let it talk to us. The Macintosh, as I learned from first-hand experience in 1984, was a huge leap forward compared to the PCs of the time. But I couldn’t have written and published the previous words and shared a digitized version of Jobs’ performance so easily, to a potential audience of 2.5 billion people, without two other inventions, the Internet and the Web.

45 years ago this year (October 29, 1969), the first ARPANET (later to be known as the Internet) link was established between UCLA and SRI. 25 years ago this year (March 1989), Tim Berners-Lee circulated a proposal for “Mesh” (later to be known as the World Wide Web) to his management at CERN.

The Internet started as a network for linking research centers. The World Wide Web started as a way to share information among researchers at CERN. Both have expanded to touch today a third of the world’s population because they have been based on open standards. The Macintosh, while a breakthrough in human-computer interaction, was conceived as a closed system and did not break from the path established by its predecessors: It was a desktop/personal mainframe. One ideology was replaced by another, with very little (and very controlled) room for outside innovation. (To paraphrase Search Engine Land’s Danny Sullivan, the big brother minions in Apple’s “1984” Super Bowl ad remind one of the people in Apple stores today).

This is not a criticism of Jobs, nor is it a complete dismissal of closed systems. It may well be that the only way for his (and his team’s) design genius to succeed was by keeping complete ownership of their proprietary innovations. But the truly breakthrough products they gave us—the iPod (and iTunes), and especially the iPhone (and “smartphones”)—were highly dependent on the availability and popularity of an open platform for sharing information, based on the Internet and the Web.

Creating a closed and proprietary system has been the business model of choice for many great inventors and some of the greatest inventions of the computer age. That’s where we were headed towards in the early 1990s: The establishment of global proprietary networks owned by a few computer and telecommunications companies, whether old (IBM, AT&T) or new (AOL). Tim Berners-Lee’s invention and CERN’s decision to offer it to the world for free in 1993 changed the course of this proprietary march, giving a new—and much expanded—life to the Internet (itself a response to proprietary systems that did not inter-communicate) and establishing a new, open platform, for a seemingly infinite number of applications and services.

As Bob Metcalfe told me in 2009: “Tim Berners-Lee invented the URL, HTTP, and HTML standards… three adequate standards that, when used together, ignited the explosive growth of the Web… What this has demonstrated is the efficacy of the layered architecture of the Internet. The Web demonstrates how powerful that is, both by being layered on top of things that were invented 17 years before, and by giving rise to amazing new functions in the following decades.”

Metcalfe also touched on the power and potential of an open platform: “Tim Berners-Lee tells this joke, which I hasten to retell because it’s so good. He was introduced at a conference as the inventor of the World Wide Web. As often happens when someone is introduced that way, there are at least three people in the audience who want to fight about that, because they invented it or a friend of theirs invented it. Someone said, ‘You didn’t. You can’t have invented it. There’s just not enough time in the day for you to have typed in all that information.’ That poor schlemiel completely missed the point that Tim didn’t create the World Wide Web. He created the mechanism by which many, many people could create the World Wide Web.”

“All that information” was what the Web gave us (and what was also on the mind of one of the Internet’s many parents, J.C.R. Licklider, who envisioned it as a giant library). But this information comes in the form of ones and zeros, it is digital information. In 2007, when Jobs introduced the iPhone, 94% of storage capacity in the world was digital, a complete reversal from 1986, when 99.2% of all storage capacity was analog. The Web was the glue and the catalyst that would speed up the spread of digitization to all analog devices and channels for the creation, communications, and consumption of information.  It has been breaking down, one by one, proprietary and closed systems with the force of its ones and zeros.

Metcalfe’s comments were first published in ON magazine which I created and published for my employer at the time, EMC Corporation. For a special issue (PDF) commemorating the 20th anniversary of the invention of the Web, we asked some 20 members of the Inforati how the Web has changed their and our lives and what it will look like in the future. Here’s a sample of their answers:

Guy Kawasaki: “With the Web, I’ve become a lot more digital… I have gone from three or four meetings a day to zero meetings per day… Truly the best will be when there is a 3-D hologram of Guy giving a speech. You can pass your hand through him. That’s ultimate.”

Chris Brogan: “We look at the Web as this set of tools that allow people to try any idea without a whole lot of expense… Anyone can start anything with very little money, and then it’s just a meritocracy in terms of winning the attention wars.”

Tim O’Reilly: “This next stage of the Web is being driven by devices other than computers. Our phones have six or seven sensors. The applications that are coming will take data from our devices and the data that is being built up in these big user-contributed databases and mash them together in new kinds of services.”

John Seely Brown: “When I ran Xerox PARC, I had access to one of the world’s best intellectual infrastructures: 250 researchers, probably another 50 craftspeople, and six reference librarians all in the same building. Then one day to go cold turkey—when I did my first retirement—was a complete shock. But with the Web, in a year or two, I had managed to hone a new kind of intellectual infrastructure that in many ways matched what I already had. That’s obviously the power of the Web, the power to connect and interact at a distance.”

Jimmy Wales: “One of the things I would like to see in the future is large-scale, collaborative video projects. Imagine what the expense would be with traditional methods if you wanted to do a documentary film where you go to 90 different countries… with the Web, a large community online could easily make that happen.”

Paul Saffo: “I love that story of when Tim Berners-Lee took his proposal to his boss, who scribbled on it, ‘Sounds exciting, though a little vague.’ But Tim was allowed to do it. I’m alarmed because at this moment in time, I don’t think there are any institutions our there where people are still allowed to think so big.”

Dany Levy (founder of DailyCandy): “With the Web, everything comes so easily. I wonder about the future and the human ability to research and to seek and to find, which is really an important skill. I wonder, will human beings lose their ability to navigate?”

Howard Rheingold: “The Web allows people to do things together that they weren’t allowed to do before. But… I think we are in danger of drowning in a sea of misinformation, disinformation, spam, porn, urban legends, and hoaxes.”

Paul Graham: “[With the Web] you don’t just have to use whatever information is local. You can ship information to anyone anywhere. The key is to have the right filter. This is often what startups make.”

How many startups have flourished on the basis of the truly great products Apple has brought to the world? And how many startups and grown-up companies today are entirely based on an idea first flashed out in a modest proposal 25 years ago? And there is no end in sight for the expanding membership in the latter camp, now also increasingly including the analogs of the world. All businesses, all governments, all non-profits, all activities are being eaten by ones and zeros. Tim Berners-Lee has unleashed an open, ever-expanding system for the digitization of everything.

We also interviewed Berners-Lee in 2009. He said that the Web has “changed in the last few years faster than it changed before, and it is crazy to for us to imagine this acceleration will suddenly stop.” He pointed out the ongoing tendency to lock what we do with computers in a proprietary jail: “…there are aspects of the online world that are still fairly ‘pre-Web.’ Social networking sites, for example, are still siloed; you can’t share your information from one site with a contact on another site.” But he remained both realistic and optimistic, the hallmarks of an entrepreneur: “The Web, after all, is just a tool…. What you see on it reflects humanity—or at least the 20 percent of humanity that currently has access to the Web… No one owns the World Wide Web, no one has a copyright for it, and no one collects royalties from it. It belongs to humanity, and when it comes to humanity, I’m tremendously optimistic.”

[Originally published on Forbes.com]

Posted in Digitization | Leave a comment

Tim O’Reilly on Open Data

[brightcove vid=3058495035001&exp3=1971702156001&surl=http://c.brightcove.com/services&pubid=1971571337001&pk=AQ~~,AAABywrPJyk~,MP34hwWOTrPs3yLiJKkINM_zsiFWIvnW&lbu=http://www.mckinsey.com/videos/video?vid=3058495035001%26plyrid=2399849255001%26Height=270%26Width=480&w=300&h=225]

Source: McKinsey

“We should define a little bit what we mean by “open,” because there’s open as in it’s open source. Anybody can take it and reuse it in whatever way they want. And I’m not sure that’s always necessary. There’s a pragmatic open and there’s an ideological open. And the pragmatic open is that it’s available. It’s available in a timely way, in a nonpreferential way, so that some people don’t get better access than others.

And if you look at so many of our apps now on the web, because they are ad-supported and free, we get a lot of the benefits of open. When the cost is low enough, it does in fact create many of the same conditions as a commons. That being said, that requires great restraint, as I said earlier, on the part of companies, because it becomes easy for them to say, “Well, actually we just need to take a little bit more of the value for ourselves. And oh, we just need a bit more of that.” And before long, it really isn’t open at all.”

Posted in Misc | Leave a comment

2013 Data Science Salary Survey: Open source tools correlate with higher salary

“In our report, 2013 Data Science Salary Survey, we make our own data-driven contribution to the conversation. We collected a survey from attendees of the Strata Conference in New York and Santa Clara, California, about tool usage and salary…

What did we find?

In a sentence: those who use data tools make more.

More specifically, the tools that correlate with higher salary are scalable and generally open source; they are often script-based or built for machine learning.  Those attendees who tend to use one such tool tend to use others––that is, these tools form a ‘cluster’ in terms of usage among our sample.  Perhaps just as interesting is that some of the traditional, popular tools such as Excel and SAS were not used as widely as R and Python. This might be food for thought for those data analysts who have thus far resisted learning how to code or moving beyond query-based data tools.”

Source: 2013 Data Science Salary Survey 

Posted in Big Data, Data Science | Leave a comment

Top 10 Predictions for $2.14 Trillion IT Market in 2014: IDC

IDC issued recently its top 10 predictions for 2014. IDC’s Frank Gens predicted that 2014 “will be about pitched battles” and a coming IT industry consolidation around a small number of big “winners.” The industry landscape will change as “incumbents will no longer be foolish enough to say we don’t compete with Amazon.”

Here’s my edited version of the predictions in the IDC press release and webcast:

Overall IT spending to grow 5.1% to $2.14 trillion, PC revenues to decline 6%

Worldwide sales of smartphones (12% growth) and tablets (18%) will continue at a “torrid pace” (accounting for over 60% of total IT market growth) at the expense of PC sales which will continue to decline. Spending on servers, storage, networks, software, and services will “fare better” than in 2013.

Android vs Apple, round 6

The Samsung-led Android community “will maintain its volume advantage over Apple,” but Apple will continue to enjoy “higher average selling prices and an established ecosystem of apps.” Google Play (Android) app downloads and revenues, however, “are making dramatic gains.” IDC advises Microsoft to “quickly double mobile developer interest in Windows.” Or else?

Amazon (and possibly Google) to take on traditional IT suppliers

Amazon Web Services’ “avalanche of platform-as-a-service offerings for developers and higher value services for businesses” will force traditional IT suppliers to “urgently reconfigure themselves.” Google, IDC predicts, will join in the fight, as it realizes “it is at risk of being boxed out of a market where it should be vying for leadership.”***

Emerging markets will return to double-digit growth of 10%

Emerging markets will account for 35% of worldwide IT revenues and, for the first time, more than 60% of worldwide IT spending growth. “In dollar terms,” IDC says, “China’s IT spending growth will match that of the United States, even though the Chinese market is only one third the size of the U.S. market.” In 2014, the number of smart connected devices shipped
in emerging markets will be almost double that shipped in developed markets and emerging markets will be a hotbed of Internet of Things market development.

In Pictures: Gartner’s 10 Strategic Technology Trends For 2013

There’s a $100 billion cloud in our future

Spending on cloud services and the technology to enable these services “will surge by 25% in 2014, reaching over $100 billion.” IDC predicts “a dramatic increase in the number of datacenters as cloud players race to achieve global scale.”

Cloud service providers will increasingly drive the IT market

As cloud-dedicated datacenters grow in number and importance, the market for server, storage, and networking components “will increasingly be driven by cloud service providers, who have traditionally favored highly componentized and commoditized designs.” The incumbent IT hardware vendors will be forced to adopt a “cloud-first” strategy, IDC predicts. 25–30% of server shipments will go to datacenters managed by service providers, growing to 43% by 2017.

Bigger big data spending

IDC predicts spending of more than $14 billion on big data technologies and services or 30% growth year-over-year, “as demand for big data analytics skills continues to outstrip supply.” The cloud will play a bigger role with IDC predicting a race to develop cloud-based platforms capable of streaming data in real time. There will be increased use by enterprises of externally-sourced data and applications and “data brokers will proliferate.” IDC predicts explosive growth in big data analytics services, with the number of providers to triple in three years. 2014 spending on these services will exceed $4.5 billion, growing by 21%.

Here comes the social enterprise

IDC predicts increased integration of social technologies into existing enterprise applications. “In addition to being a strategic component in virtually all customer engagement and marketing strategies,” IDC says, “data from social applications will feed the product and service development process.” By 2017, 80% of Fortune 500 companies will have an active customer community, up from 30% today.

Here comes the Internet of Things

By 2020, the Internet of Things will generate 30 billion autonomously connected end points and $8.9 trillion in revenues. IDC predicts that in 2014 we will see new partnerships among IT vendors, service providers, and semiconductor vendors that will address this market. Again, China will be a key player:  The average Chinese home in 2030 will have 40–50 intelligent devices/sensors, generating 200TB of data annually.

The digitization of all industries

By 2018, 1/3 of share leaders in virtually all industries will be “Amazoned” by new and incumbent players. “A key to competing in these disrupted and reinvented industries,” IDC says, “will be to create industry-focused innovation platforms (like GE’s Predix) that attract and enable large communities of innovators – dozens to hundreds will emerge in the next several years.” Concomitant with this digitization of everything trend, “the IT buyer profile continues to shift to business executives. In 2014, and through 2017, IT spending by groups outside of IT departments will grow at more than 6% per year.”

***Can’t resist quoting my August 2011 post: “Consumer vs. enterprise is an old and soon-to-be obsolete distinction. If Google will not take away some of Microsoft’s (and IBM’s, etc. for that matter) “enterprise” revenues, someone else will. At stake are the $1.5 trillion spent annually by enterprises on hardware, software, and services. If you include what enterprises spend on IT internally (staff, etc.), you get at least $3 trillion. A big chunk of that will move to the cloud over the next fifteen years. Compare this $3 trillion to the $400 billion spent annually on all types of advertising worldwide.  Why leave money on the table?”

[Originally published on Forbes.com]

Posted in Big Data, Internet of Things | Leave a comment

The Digitization of Everything or the Coming of the Internet of Flying Things

My post on Inside Tech Talk:

The only bit of news—and a lot of buzz—that came out of Jeff Bezos’ interview with Charlie Rose on CBS’ 60 Minutes last Sunday was the unveiling of Prime Air. It’s an Amazon “R&D project” that is investigating the delivery of packages (up to five pounds) by aerial drones, getting them to the customer’s door in 30 minutes or less.

On its website, Amazon says that “We hope the FAA’s rules will be in place as early as sometime in 2015. We will be ready at that time.” Indeed, the FAA promises to be ready by 2015 according to its “roadmap for integration of civil unmanned aircraft systems.” But in the interview, Bezos refused to commit to a specific date, talking about the work they still need to do to ensure reliability and redundancy.

Reliability is a great challenge today for octocopters (the type of drones Amazon will use) as can be seen in this YouTube video. And even if Amazon finds out how to reduce the risk to a minimum, it will always be there, to say nothing about the opposition from privacy advocates, noise complainers, and others. So why is Bezos investing in the future (or fiction) of delivery?

Some explain it as a gimmick aimed to distract investors’ attention from Amazon’s latest financial report, as if the lack of profits is anything new to Amazon’s investors. Others may see it as Google-envy, one-upping its rival by adding “flying” to “autonomous vehicles,” as if Bezos has suddenly metamorphosed into a Brin.

I see it as a logical extension of the focus on the speed of delivery that has been driving Bezos and Amazon for eighteen years. As Bezos explained to Wired’s Steven Levy when he was asked about the link between retail and Amazon’s new thrust into consumer electronics: “…we’ve always focused on reducing the time between order and delivery. In hardware, it’s the same principle. An example is the time between when we take delivery on a processor to when it’s being used in a device by a customer.”

Bezos is bothered by waste and inefficiency the same way Steve Jobs was bothered by lack of imagination or bad aesthetics.

Product design was the core of Steve Jobs’ strategy to ride the digitization of everything. The speed of delivery is the core of Bezos’. With drone delivery, he is trying to eliminate the wasteful irritation of the last mile. No matter how close to where his customers live he is going to build Amazon’s warehouses (36 and counting), the most he can promise is same-day delivery. So he is going to circumvent and disrupt UPS and FedEx, the delivery partners that have been helped so much by Amazon’s success over the last decade. In the future, he may even disrupt his own drone delivery, by completely digitizing the last mile, through 3D printing in the customer’s house.

In the 60 Minutes interview, Bezos explained everything he does by fear of disruption. He believes that Amazon, like so many other companies in the past, will be disrupted one day and will simply disappear. He just hopes that will happen after he dies. In the meantime, he rides the digitization of everything with a maniacal focus on the speed of delivery.

Update: See here for Fred Smith of FedEx talking in 2009 (!) about their desire to use drones. HT @daviottenheimer

Posted in Digitization | Leave a comment

Mobile Drives Big Data: Ericsson Mobility Report

MOBILE SUBSCRIPTIONS OUTLOOK
MOBILE TRAFFIC
MOBILE APPLICATION TRAFFIC OUTLOOK

The complete report is here and here

Posted in Big Data | Leave a comment

Sources and Types of Big Data (Infographic)

INTELLIGENCE BY VARIETY
Posted in Big Data | Leave a comment

Big Data: Who, Why, and How (Infographic)

The who, why and how of BIG DATA

“Early adopters of Big Data analytics have gained a significant lead over the rest of the corporate world. Examining more than 400 large companies, we found that those with the most advanced analytics capabilities are outperforming competitors by wide margins.”

Source: Bain & Company

Posted in Big Data, Statistics | Leave a comment

DataKind’s Jack Porway on Data Science

[youtube=http://www.youtube.com/watch?v=Mm1RplOU0cQ&w=560&h=315]

“If you leave an excited data scientist on his own to solve a problem, he’s going to solve his own problem – which is usually parking his car, or finding a bar to drink at. The trick that we worked on was actually less about data and more about translation, about finding a way for data scientists to speak the language of the people who were trying to solve the big problems… the biggest [challenge] is actually the framing of the problem: really finding the question. As any good data scientist will tell you, it’s not so much about the data, it’s the question you start with”–Jack Porway, DataKind

More here

Posted in Data Science | Leave a comment

What Will Make You a Big Data Leader?

What Will Make You a Big Data Leader?

The IBM Institute for Business Value’s 2013 analytics survey surveyed 900 business and IT executives from 70 countries. “Leaders” (19% of the sample) were respondents self identified as “substantially outperforming their market or industry peers” in a question used by the IBM Institute for Business Value for years across a wide variety of surveys.

The full report is here

Posted in Big Data, Statistics | Leave a comment