Information Overload?

It is now clear that what experts suspected for years is in fact true -- that the NSA monitors every form of electronic communication around the globe. This fact raises an important question: How can an intelligence agency, even one as large and well-staffed as the NSA with its 40,000 employees, work meaningfully with such a flood of information?

The answer to this question is part of a phenomenon that is currently a major topic for the business community as well and goes by the name "Big Data." Thanks to new database technologies, it is now possible to connect entirely disparate forms of data and analyze them automatically.

A rare glimpse into what intelligence services can do by applying this "big data" approach came last year from David Petraeus. This new form of data analysis is concerned with discovering "non-obvious relationships," the then freshly minted CIA director explained at a conference. This includes, for example "finding connections between a purchase here, a phone call there, a grainy video, customs and immigration information."

The goal, according to Petraeus, is for big data to "lead to automated discovery, rather than depending on the right analyst asking the right question." Algorithms pick out connections automatically from the unstructured sea of data they trawl. "The CIA and our intelligence community partners must be able to swim in the ocean of 'Big Data.' Indeed, we must be world class swimmers -- the best, in fact," the CIA director continued.

The Surveillance State

The value of big data analysis for US intelligence agencies can be seen in the amount the NSA and CIA are investing in it. Not only does this include multimillion-dollar contracts with providers specializing in data mining services, but the CIA also invests directly, through its subsidiary company In-Q-Tel, in several big data start-ups.

It's about rendering people and their behavior predictable. The NSA's research projects aim to forecast, on the basis of telephone data and Twitter and Facebook posts, when uprisings, social protests and other events will occur. The agency is also researching new methods of analysis for surveillance videos with the hopes of recognizing conspicuous behavior before an attack is committed.

Gus Hunt, the CIA's chief technology officer, made a forthright admission in March: "We fundamentally try to collect everything and hang onto it forever." What he meant by "everything," Hunt also made clear: "It is really very nearly within our grasp to be able to compute on all human-generated information," he said.

That statement is difficult to reconcile with the Fourth Amendment to the US Constitution, which guarantees the right to privacy. This is probably why Hunt added, almost apologetically: "Technology in this world is moving faster than government or law can keep up."