From the May 26 issue of the Economist: “In A short story called ‘On Exactitude in Science’, Jorge Luis Borges described an empire in which cartographers became so obsessive that they produced a map as big as the empire itself. This was so cumbersome that future generations left it to disintegrate. (‘[I]n the western deserts, tattered fragments of the map are still to be found, sheltering some occasional beast or beggar.’)
As usual, the reality of the digital age is outpacing fiction. Last year people stored enough data to fill 60,000 Libraries of Congress. The world’s 4 billion mobile-phone users (12% of whom own smartphones) have turned themselves into data-streams. YouTube claims to receive 24 hours of video every minute. Manufacturers have embedded 30m sensors into their products, converting mute bits of metal into data-generating nodes in the internet of things. The number of smartphones is increasing by 20% a year and the number of sensors by 30%…
Big data has the same problems as small data, but bigger. Data-heads frequently allow the beauty of their mathematical models to obscure the unreliability of the numbers they feed into them. (Garbage in, garbage out.) They can also miss the big picture in their pursuit of ever more granular data…
The sheer size of today’s data banks means that companies need to be more careful than ever to treat data as a slave rather than a master. There is no substitute for sound intuition and wise judgment. But if firms can preserve a little scepticism, they can surely squeeze important insights from the ever-growing store of data. In the 1980s and 1990s retailers such as Walmart used their mastery of retailing data to launch the “big-box” revolution (huge out-of-town stores with ultra-low prices). Today’s big data will provide the raw material for further revolutions.”
