What do you think has been named the ‘sexiest job of the 21st century’? International footballer? Supermodel? Brain surgeon?
All wrong. The answer is in fact ‘big data scientist‘. This does not mean a data scientist who happens to be six foot six and 20 stone. It refers to those who can make sense of the vast and constantly expanding heap of digital data. The amount of data that is gathered is astonishing.
According to IBM, we create 2.5 quintillion bytes of data every day. This means that 90% of the data in the world today has been created in the last two years alone.
Where does it all come from? The constant use of the internet generates data all the time. Every time you use your mobile phone, you create data. But human interaction is not the only cause. There is the ‘internet of things’.
Two million security cameras in the UK record images that are faithfully filed away. Sensors also give out stores of data, such as climate information and detecting possible faults on aeroplanes and cars. At home we could even have sensors which tell us when to turn on the radiator or even restock the fridge.
Who is Using All This Data?
Two of the biggest users of data are financial trading firms and medical researchers. Many hedge funds rely upon high powered computers to spot tiny price anomalies. By acting instantly, they can take advantage of these discrepancies to turn a profit.
Proximity to a data centre is famously important. Even the few milliseconds that it takes for data to pass down a cable can mean the difference between profit and loss.
Medical research similarly depends upon data like never before. Dramatic improvements in sequencing machines have made it possible to read DNA rapidly and cheaply. All over the world researchers are searching through vast quantities of DNA to look for links to diseases.
But it is in the cut-throat world of business that data mining is seen as critical to success. Every time you go the supermarket and use your loyalty card, you reveal a little more about yourself. You will have noticed that the offers you receive become increasingly relevant and attractive.
Online we reveal even more. When we checkout at the supermarket we reveal only what we have bought. But when online, we reveal everything we have looked at.
There is something undeniably sinister about this. Google and other internet giants are often accused of knowing too much about us and using the information for nefarious purposes. Of much greater concern are the intentions of the government.
Governments use data monitoring in the fight against crime and terrorism. CCTV cameras have probably done more to reduce crime than anything, and many a terrorist plot has been foiled by the listening ear of the intelligence services. But the boundary between this legitimate activity and infringement of privacy is a fine line.
The New Rock Stars of the Tech World
Data can be used to profile customers and sell them things. Data can be used for useful medical research and ‘socially useless’ financial trading. But it can only be used if we make some sense of it all.
‘Big data‘ refers not to data routinely and legitimately collected, but also to all unstructured ‘dark data’. This can include email archives, warranty forms, call centre recordings and doctors’ notes, all of which could contain nuggets of useful information.
To find it, the big data scientists use algorithms, which are ‘effective methods for solving a problem expressed as a finite sequence of steps.‘These formulas now have real value. Netflix paid $1m for an algorithm that more accurately predicts which films a customer would like.
The masters of the algorithms are now the rock stars of the tech world. ‘The rise in the importance of algorithms,’ says computer science professor Dr András Faragó, ‘parallels the earlier ascendance of software itself, which once played a secondary role to the original star, hardware.‘
According to Gartner, 42% of big businesses have adopted big data technologies and by 2016 30% will be wielding their information assets also as a currency – bartering or trading with them, or even outright selling them.
The Best Bet for Investors
Clearly there is value in the interpretation of data, but for investors the best bet is data storage. Value or no value, the data must be stored somewhere and according to the International Data Corporation, ‘storage is increasing at a compound annual growth rate of 53%… revenue from storage consumed by Big Data & Analytics environments will increase from $379.9m in 2011 to nearly $6bn in 2016.‘
Tom Bulford
Contributing Editor, Money Morning
Publisher’s Note: This article originally appeared in MoneyWeek
From the Archives…
Why Small-Cap Resource Stocks Beat Blue Chips Hands Down
10-05-2013 – Dr Alex Cowie
Can Australian Stocks Defy Gravity if The Australian Dollar Falls?
9-05-2013 – Murray Dawes
Build Wealth Fast through the Resource Sector
8-05-2013 – Dr Alex Cowie
36% Potential Upside for Australian Stocks Over the Next Two Years
7-05-2013 – Kris Sayce
The Key to Becoming a Successful Investor
6-05-2013 – Kris Sayce
