At first glance, the story always appears to be the same. A needle has disappeared into the haystack -- information lost in a sea of data.
For some time now, though, it appears America's intelligence services have been trying to tackle the problem from a different angle. "If you're looking for a needle in the haystack, you need a haystack," says Jeremy Bash, the former chief of staff to ex-CIA head Leon Panetta.
An enormous haystack it turns out -- one comprised of the billions of minutes of daily cross-border telephone traffic. Add to that digital streams from high-bandwidth Internet cables that transport data equivalent to that held in Washington's Library of Congress around the world in the course of a few seconds. And then add to that the billions of emails sent to international destinations each day -- a world of entirely uncontrolled communication. And also a world full of potential threats -- at least from the intelligence services' perspective. Those are the "challenges," an internal statement at the National Security Agency (NSA), the American signals intelligence organization, claims.
Four-star General Keith Alexander -- who is today the NSA director and America's highest-ranking cyber warrior as the chief of the US Cyber Command -- defined these challenges. Given the cumulative technological eavesdropping capacity, he asked during a 2008 visit to Menwith Hill, Britain's largest listening station near Harrogate in Yorkshire, "Why can't we collect all the signals all the time?"
All the signals all the time. Wouldn't that be the NSA's ideal haystack? So what would the needle be? A trail to al-Qaida, an industrial facility belonging to an enemy state, plans prepared by international drug dealers or even international summit preparations being made by leading politicians of friendly nations? Whatever the target, it would be determined on a case by case basis. What is certain, however, is that there would always be a haystack.
A Fiasco for the NSA
Just how close America's NSA got to this dream in cozy cooperation with other Western intelligence services has been exposed in recent weeks by a young American who, going by outward appearances, doesn't look much like the hero he is being celebrated as around the world by people who feel threatened by America's enormous surveillance apparatus.
The whole episode is a fiasco for the NSA which, in contrast to the CIA, has long been able to conduct its spying without drawing much public attention. Snowden has done "irreversible and significant damage" to US national security, Alexander told ABC a week ago. Snowden's NSA documents contain more than one or two scandals. They are a kind of digital snapshot of the world's most powerful intelligence agency's work over a period of around a decade. SPIEGEL has seen and reviewed a series of documents from the archive.
The documents prove that Germany played a central role in the NSA's global surveillance network -- and how the Germans have also become targets of US attacks. Each month, the US intelligence service saves data from around half a billion communications connections from Germany.
No one is safe from this mass spying -- at least almost no one. Only one handpicked group of nations is excluded -- countries that the NSA has defined as close friends, or "2nd party," as one internal document indicates. They include the UK, Australia, Canada and New Zealand. A document classified as "top secret" states that, "The NSA does NOT target its 2nd party partners, nor request that 2nd parties do anything that is inherently illegal for NSA to do."
'We Can, and Often Do Target Signals'
For all other countries, including the group of around 30 nations that are considered to be 3rd party partners, however, this protection does not apply. "We can, and often do, target the signals of most 3rd party foreign partners," the NSA boasts in an internal presentation.
According to the listing, Germany is among the countries that are the focus of surveillance. Thus, the documents confirm what had already been suspected for some time in government circles in Berlin -- that the US intelligence service, with approval from the White House, is spying on the Germans -- possibly right up to the level of the chancellor. So it comes as little surprise that the US has used every trick in the book to spy on the Washington offices of the European Union, as one document viewed by SPIEGEL indicates.
But the new aspect of the revelations isn't that countries are trying to spy on each other, eavesdropping on ministers and conducting economic espionage. What is most important about the documents is that they reveal the possibility of the absolute surveillance of a country's people and foreign citizens without any kind of effective controls or supervision. Among the intelligence agencies in the Western world, there appears to be a division of duties and at times extensive cooperation. And it appears that the principle that foreign intelligence agencies do not monitor the citizens of their own country, or that they only do so on the basis of individual court decisions, is obsolete in this world of globalized communication and surveillance. Britain's GCHQ intelligence agency can spy on anyone but British nationals, the NSA can conduct surveillance on anyone but Americans, and Germany's BND foreign intelligence agency can spy on anyone but Germans. That's how a matrix is created of boundless surveillance in which each partner aids in a division of roles.
The documents show that, in this situation, the services did what is not only obvious, but also anchored in German law: They exchanged information. And they worked together extensively. That applies to the British and the Americans, but also to the BND, which assists the NSA in its Internet surveillance.
SPIEGEL has decided not to publish details it has seen about secret operations that could endanger the lives of NSA workers. Nor is it publishing the related internal code words. However, this does not apply to information about the general surveillance of communications. They don't endanger any human lives -- they simply describe a system whose dimensions go beyond the imaginable. This kind of global debate is actually precisely what Snowden intended and what motivated his breach of secrecy. "The public needs to decide whether these policies are right or wrong," he says.
The facts, which are now a part of the public record thanks to Snowden, disprove the White House's line of defense up until now, which has been that the surveillance is necessary to prevent terrorist attacks, as President Barack Obama said during his recent visit to Berlin. NSA chief Alexander has sought to justify himself by saying that the NSA has prevented 10 terrorist attacks in the United States alone. Globally, he says that 50 terrorist plots have been foiled with the NSA's help. That may be true, but it is difficult to verify and at best only part of the truth.
Research in Berlin, Brussels and Washington, as well as the documents that have been reviewed by the journalists at this publication, reveal how overreaching the US surveillance has been.
Germany, for its part, has a central role in this global spying system. As the Guardian newspaper, which is working together with Snowden, recently revealed, the NSA has developed a program for the incoming streams of data called "Boundless Informant." The program is intended to process connection data from all incoming telephone calls in "near real time," as one document states. It doesn't record the contents of the call, just the metadata -- in other words, the phone numbers involved in the communication.
It is precisely the kind of data retention that has been the subject of bitter debate in Germany for years. In 2010, the Federal Constitutional Court in Karlsruhe even banned the practice.
"Boundless Informant" produces heat maps of countries in which the data collected by the NSA originates. The most closely monitored regions are located in the Middle East, followed by Afghanistan, Iran and Pakistan. The latter two are marked in red on the NSA's map of the world. Germany, the only country in Europe on the map, is shown in yellow, a sign of considerable spying.
Spying on the European Union
An NSA table (see graphic), published for the first time here by SPIEGEL, documents the massive amount of information captured from the monitored data traffic. According to the graph, on an average day last December, the agency gathered metadata from some 15 million telephone connections and 10 million Internet datasets. On Dec. 24, it collected data on around 13 million phone calls and about half as many Internet connections.
On the busiest days, such as Jan. 7 of this year, the information gathered spiked to nearly 60 million communications processes under surveillance. The Americans are collecting metadata from up to half a billion communications a month in Germany -- making the country one of the biggest sources of streams of information flowing into the agency's gigantic sea of data.
Another look at the NSA's data hoard shows how much less information the NSA is taking from countries like France and Italy. In the same period, the agency recorded data from an average of around 2 million connections, and about 7 million on Christmas Eve. In Poland, which is also under surveillance, the numbers varied between 2 million and 4 million in the first three weeks of December.
But the NSA's work has little to do with classic eavesdropping. Instead, it's closer to a complete structural acquisition of data. Believing that less can be extrapolated from such metadata than from intercepted communication content would be a mistake, though. It's a gold mine for investigators, because it shows not only contact networks, but also enables the creation of movement profiles and even predictions about the possible behavior of the people participating in the communication under surveillance.
According to insiders familiar with the German portion of the NSA program, the main interest is in a number of large Internet hubs in western and southern Germany. The secret NSA documents show that Frankfurt plays an important role in the global network, and the city is named as a central base in the country. From there, the NSA has access to Internet connections that run not only to countries like Mali or Syria, but also to ones in Eastern Europe. Much suggests that the NSA gathers this data partly with and without Germany's knowledge, although the individual settings by which the data is filtered and sorted have apparently been discussed. By comparison, the "Garlick" system, with which the NSA monitored satellite communication out of the Bavarian town of Bad Aibling for years, seems modest. The NSA listening station at Bad Aibling was at the center of the German debate over America's controversial Echelon program and alleged industrial espionage during the 1990s.
"The US relationship with Germany has been about as close as you can get,"American journalist and NSA expert James Bamford recently told German weekly Die Zeit. "We probably put more listening posts in Germany than anyplace because of its proximity to the Soviet Union."
Such foreign partnerships, one document states, provide "unique target access."
'Privacy of Telecommunications' Is 'Inviolable'
But the US does not share the results of the surveillance with all of these foreign partners, the document continues. In many cases, equipment and technical support are offered in exchange for the signals accessed. Often the agency will offer equipment, training and technical support to gain access to its desired targets. These "arrangements" are typically bilateral and made outside of any military and civil relationships the US might have with these countries, one top secret document shows. This international division of labor seems to violate Article 10 of Germany's constitution, the Basic Law, which guarantees that "the privacy of correspondence, posts and telecommunications shall be inviolable" and can only be suspended in narrowly defined exceptions.
"Any analyst can target anyone anytime," Edward Snowden said in his video interview, and that includes a federal judge or the president, if an email address is available, he added.
Just how unscrupulously the US government allows its intelligence agencies to act is documented by a number of surveillance operations that targeted the European Union in Brussels and Washington, for which it has now become clear that the NSA was responsible.
A little over five years ago, security experts discovered that a number of odd, aborted phone calls had been made around a certain extension within the Justus Lipsius building, the headquarters of the European Council, the powerful body representing the leaders of the EU's 27 member states. The calls were all made to numbers close to the one used as the remote servicing line of the Siemens telephone system used in the building. Officials in Brussels asked the question: How likely is it that a technician or service computer would narrowly misdial the service extension a number of times? They traced the origin of the calls -- and were greatly surprised by what they found. It had come from a connection just a few kilometers away in the direction of the Brussels airport, in the suburb of Evere, where NATO headquarters is located.
The EU security experts managed to pinpoint the line's exact location -- a building complex separated from the rest of the headquarters. From the street, it looks like a flat-roofed building with a brick facade and a large antenna on top. The structure is separated from the street by a high fence and a privacy shield, with security cameras placed all around. NATO telecommunications experts -- and a whole troop of NSA agents -- work inside. Within the intelligence community, this place is known as a sort of European headquarters for the NSA.
A review of calls made to the remote servicing line showed that it was reached several times from exactly this NATO complex -- with potentially serious consequences. Every EU member state has rooms at the Justus Lipsius building for use by ministers, complete with telephone and Internet connections.
Unscrupulous in Washington
The NSA appears to be even more unscrupulous on its home turf. The EU's diplomatic delegation to the United States is located in an elegant office building on Washington's K Street. But the EU's diplomatic protection apparently doesn't apply in this case. As parts of one NSA document seen by SPIEGEL indicate, the NSA not only bugged the building, but also infiltrated its internal computer network. The same goes for the EU mission at the United Nations in New York. The Europeans are a "location target," a document from Sept. 2010 states. Requests to discuss these matters with both the NSA and the White House went unanswered.
Now a high-level commission of experts, agreed upon by European Justice Commissioner Viviane Reding and US Attorney General Eric Holder, is to determine the full scope of the routine data snooping and discuss the legal protection possibilities for EU citizens. A final report is expected to be released in October.
The extent of the NSA's systematic global surveillance network is highlighted in an overview from Fort Meade, the agency's headquarters. It describes a number of secret operations involving the surveillance of Internet and international data traffic. "In the Information Age, (the) NSA aggressively exploits foreign signals traveling complex global networks," an internal description states.
Details in a further, previously unpublished document reveal exactly what takes place there. It describes how the NSA received access to an entire bundle of fiber-optic cables, which have a data-transfer capacity of several gigabytes per second. It is one of the Internet's larger superhighways. The paper indicates that access to the cables is a relatively recent development and includes Internet backbone circuits, "including several that service the Russian market." Technicians in Fort Meade are able to access "thousands of trunk groups connected worldwide," according to the document. In a further operation, the intelligence organization is able to monitor a cable that collects data flows from the Middle East, Europe, South America and Asia (see graphic).
But it is not just intelligence agencies from allied nations that have willingly aided the NSA. Revelations related to the Prism program make it clear that agents likewise access vast quantity of data from US Internet companies.
NSA 'Alliances With Over 80 Major Global Corporations'
Heads of these companies have vociferously denied that the NSA has direct access to their data. But it would seem that, outside of the Prism program, dozens of companies have willingly worked together with the US intelligence agency.
According to the documents seen by SPIEGEL, a particularly valuable partner is a company which is active in the US and has access to information that crisscrosses America. At the same time, this company, by virtue of its contacts, offers "unique access to other telecoms and (Internet service providers)." The company is "aggressively involved in shaping traffic to run signals of interest past our monitors," according to a secret NSA document. The cooperation has existed since 1985, the documents say.
Apparently, it's not an isolated case, either. A further document clearly demonstrates the compliance of a number of different companies. There are "alliances with over 80 major global corporations supporting both missions," according to a paper that is marked top secret. In NSA jargon, "both missions" refers to defending networks in the US, on the one hand, and monitoring networks abroad, on the other. The companies involved include telecommunications firms, producers of network infrastructure, software companies and security firms.
Such cooperation is an extremely delicate issue for the companies involved. Many have promised their customers data confidentiality in their terms and conditions. Furthermore, they are obliged to follow the laws of the countries in which they do business. As such, their cooperation deals with the NSA are top secret. Even in internal NSA documents, they are only referred to using code names.
"There has long been a very close and very secret relationship between a number of telecoms and the NSA," Bamford, the expert on the NSA, told Die Zeit. "Every time it gets discovered it stops for a while and then starts up again."
The importance of this rather peculiar form of public-private partnership was recently made clear by General Alexander, the NSA chief. At a technology symposium in a Washington, DC, suburb in May, he said that industry and government must work closely together. "As great as we have it up there, we cannot do it without your help," he said. "You know, we can't do our mission without the great help of all the great people here." If one believes the documents, several experts were sitting in the audience from companies that had reached a cooperation deal with the NSA.
In the coming weeks, details relating to the collaboration between Germany's BND and the NSA will be the focus of a parliamentary investigative committee in Berlin responsible for monitoring the intelligence services. The German government has sent letters to the US requesting additional information. The questions that need to be addressed are serious. Can a sovereign state tolerate a situation in which half a billion pieces of data are stolen on its territory each month from a foreign country? And can this be done especially when this country has identified the sovereign state as a "3rd party foreign partner" and, as such, one that can be spied on at any time, as has now become clear?
So far, the German government has made nothing more than polite inquiries. But facts that have now come to light will certainly increase pressure on Chancellor Angela Merkel and her government. Elections, after all, are only three months away, and Germans -- as Merkel well knows -- are particularly sensitive when it comes to data privacy.
The NSA's Library of Babel
In a story written by the blind writer Jorge Luis Borges, the Library of Babel is introduced as perhaps the most secretive of all labyrinths: a universe full of bookshelves connected by a spiral staircase that has no beginning and no end. Those inside wander through the library looking for the book of books. They grow old inside without ever finding it.
If an actual building could really approach this imaginary library, it is the structure currently being erected in the Utah mountains near the city of Bluffdale. There, on Redwood Road, stands a sign with black letters on a white background next to a freshly paved road. Restricted area, no access, it reads. In Defense Department documents, form No. 1391, page 134, the buildings behind the sign are given the project No. 21078. It refers to the Utah Data Center, four huge warehouses full of servers costing a total of €1.2 billion ($1.56 billion).
Built by a total of 11,000 workers, the facility is to serve as a storage center for everything that is captured in the US data dragnet. It has a capacity that will soon have to be measured in yottabytes, which is 1 trillion terabytes or a quadrillion gigabytes. Standard external hard drives sold in stores have a capacity of about 1 terabyte. Fifteen such hard drives could store the entire contents of the Library of Congress.
The man who first made information about the Utah center public, and who likely knows the most about the NSA, is James Bamford. He says: "The NSA is the largest, most expensive and most powerful intelligence agency in the world."
Since the 9/11 terror attacks, the NSA's workforce has steadily grown and its budget has constantly increased. SPIEGEL was able to see confidential figures relating to the NSA that come from Snowden's documents, though the statistics are from 2006. In that year, 15,986 members of the military and 19,335 civilians worked for the NSA, which had an annual budget of $6.115 billion. These numbers and more recent statistics are officially confidential.
In other words, there is a good reason why NSA head Keith Alexander is called "Emporer Alexander." "Keith gets whatever he wants," says Bamford.
Still, Bamford doesn't believe that the NSA completely fulfills the mission it has been tasked with. "I've seen no indications that NSA's vastly expanded surveillance has prevented any terrorist activities," he says. There is, however, one thing that the NSA managed to predict with perfect accuracy: where the greatest danger to its secrecy lies. In internal documents, the agency identifies terrorists and hackers as being particularly threatening. Even more dangerous, however, the documents say, is if an insider decides to blow the whistle.
An insider like Edward Joseph Snowden.
REPORTED BY LAURA POITRAS, MARCEL ROSENBACH, FIDELIUS SCHMID, HOLGER STARK AND JONATHAN STOCK