Big Data as Big Brother

Suddenly, "big data" doesn't seem so great, at least not when "big brother" is using it to keep tabs on the American people.

Last year I noted the market researchers at IDC were saying big data technology and services would grow from a rather mundane $3 billion cottage industry in 2010 to almost $17 billion by 2015 -- a 40% compounded annual growth rate featuring 7.9 zettabytes of data stored globally. Yet I also pointed out that a study by OpTier said most businesses won't be able to take advantage of it and 85% "will fail to effectively exploit big data for competitive advantage."

Apparently that's because most don't have the resources of the federal government, which, as we've just uncomfortably found out, is taking a keen interest in our phone records -- at least if you're a Verizon customer -- as well as the information housed by a bevy of Silicon Valley stalwarts including Google (GOOGL 1.27%), Apple, and Facebook (META 2.98%).

It all seems so quaint now when thinking back that our biggest problem may have been a status update our employer might read. Today, it's considerably more worrisome that the government thinks it's OK to collect your private phone records to know whom you called, when you called, and how long you were talking. While President Obama assured us no one's listening in on our phone conversations, the National Security Agency has already admitted it "inadvertently" collected emails and phone calls of private individuals.

The risk here for business -- aside from the obvious Orwellian implications of this massive level of government snooping -- is the public may no longer think the vast collection of data by private enterprise is so benign. What might be a huge investment opportunity could instead be turned into a pariah, and businesses associated with the practice of collecting and processing the massive amounts of data could be shunned by the investing world for engaging in it.

IBM (IBM 0.16%), an early and important first mover in big data, says we create 2.5 quintillion bytes of data every single day. In its first quarter, business analytics revenue grew 7% as the business analytics content within its global business services division jumped by double-digit percentages. It's been making a number of acquisitions in the space as well, spending $51 million in the first quarter along to add StoredIQ and StarAnalytics to its roster of services offering big data initiatives.

Similarly, recently public Splunk (SPLK), which describes itself as the "leading software platform for real-time operational intelligence," enjoyed revenue growth of 54% in the first quarter as it signed up more than 350 new enterprise customers that look to it to process and analyze the data dump their receiving.

Right now these companies are still seen as investment-grade opportunities, but as the snooping scandal widens, investors may just begin to cast a dark eye their way as well. We already think it's a bit creepy how Google "knows" the kinds of ads to display on our web pages through AdWords, but it also just released a white paper showing the predictive capabilities of which movies will be hits based on "the timing and category of Google searches and paid clicks."

In all honesty, I don't think the public at large will make that leap between the government's spying on us and what big business is using big data for because it has a short memory. How many times has Facebook become embroiled in a "privacy" spat with its users, yet its user base continues to grow? Revenue jumped 37% in the first quarter and daily active users soared 26% to 665 million (monthly users were up 23% to 1.11 billion).

So if you've bought shares in large, established big data players like IBM and Oracle, or even smaller upstarts like Splunk and TIBCO Software, don't worry about selling the stock any time soon. Should more damning news like the PRISM program be revealed, these stocks could be cast in a bad light. But since we're unwilling to stop broadcasting our most intimate details on the web, there's going to be a market for that data and for companies to process and analyze it. And perhaps even get in bed with the government to facilitate it.