On February 20th, Berlin actress Jennifer Ulrich posted a video on her Facebook page documenting events in Clausnitz, a small town in the eastern state of Saxony. The video shows residents surrounding a bus filled with refugees and police pulling the terrified passengers out in order to protect them from the angry mob. "I am very ashamed right now of being German when I see the images from Clausnitz," Ulrich wrote. "My blood freezes in my veins when I see such inhumanity and hatred."
Ulrich has had a Facebook page since May 2013. She generally shares photos from films she's working on or from public appearances and adds a nice comment. Those interested in her posts tend to be either colleagues or fans.
But two days after the Clausnitz post, a Facebook user going by the pseudonym Mario Weber posted a death threat on the actress' page. "They should take a chainsaw to your shit-ugly feces face," Weber wrote. "Ulrich, your life is worthless, just die." Another post followed a few minutes later, no less direct. "The day of reckoning will come, leftist scum, and then I will be there to slaughter each and every one of you in the bloodiest way possible."
Ulrich, who has had roles in some major German movies, doesn't usually take criticism terribly seriously. It's not uncommon for people to respond snidely to her posts. Normally she just ignores them. This time, though, after contacting her agent, the actress decided to defend herself. That evening she contacted the police over the Internet and made a formal request for the incident to be investigated. She also reported the posting to Facebook, requesting that the poster be blocked.
'We Want People To Feel Safe'
Facebook promises its members that it will take action against all users who harass or threaten others. "We want people to feel safe when using Facebook," the social media platform's "Community Standards" page states.
In these standards, the company pledges to carefully review reports of "threatening language" and remove "credible threats of physical harm." Posts with the aim of shaming or degrading private individuals are also prohibited. "We don't tolerate bullying or harassment," it states under the heading "Helping to Keep you Safe."
Two days after Ulrich reported the post about sawing her face up with a chainsaw, she received a response that the post in question had been reviewed. The screening had determined that the post had "not violated our community standards."
The company provided no further explanation. Nor was it evident who had conducted the review or who had answered her once it had been completed. Unlike other companies, Facebook declines to provide names when interacting with users. As Ulrich would discover, there wasn't even a telephone number to call with follow-up questions.
Normally, the story would end here, as is the case with so many users who discover hateful comments on their Facebook pages. But Ulrich can be stubborn and doesn't give in easily when something upsets her. She posted the answer from Facebook on her page together with the question, "I wonder what this insanely friendly, apparently right-wing user would have to write in order for Facebook to deem his comments worthy of deletion?"
This time, the company identified a violation of its terms. Ulrich received a message from Facebook that her posting had been deleted and she was admonished to acquaint herself with Facebook's community standards. Then her page was blocked. When she tried to send a note to a friend, a message popped up on her screen reading, "Your account is temporarily unavailable." Later, the "Facebook Team" would explain that the deletion had been an "error". But a few days prior to that about-face, Germany's most famous actor, Til Schweiger, had intervened and taken the incident public. A tabloid newspaper in Berlin also picked up the story.
Each day, millions of people post on Facebook -- whatever it is, at any given moment, that amuses, annoys, pleases or otherwise occupies them. The social network has 29 million users in Germany alone. For many, it is no longer just the platform they use to contact their friends -- it is also their primary source of information. One out of four people use the network to get their news. Facebook has become their window on the world.
People sharing all possible interests can be found on the site today: hippies, animal lovers, people seeking to find their inner self -- but also less nice people who consider Adolf Hitler to have been the greatest statesman of all time and believe that tree huggers are evil incarnate. It's one thing for a person to privately venerate Hitler, but it's quite another when they believe they need to share their opinion with the world. That's where the problem starts.
In Germany, a high value is placed on freedom of expression, but it is not without limits -- a product of the country's murderous, 20th century history. Those who insult or defame others open themselves to lawsuits and damage payments. Some statements may even prompt prosecutors to file criminal charges. Holocaust denial, for example, is a prosecutable offense in Germany. Inciting the masses -- such as the incitement of hatred against individual groups -- or the glorification of National Socialism are also prohibited in the country.
The government is often not shy when it comes to enforcement. Horst Mahler, a former member of the Red Army Faction, a notorious left-wing German terrorist group active in the 1970s and '80s, who veered to the extreme right later in life, was imprisoned in 2009 for incitement. He was sentenced to 10 years in prison for, among other things, giving a Hitler salute and denying the Holocaust. Some perpetrators of manslaughter get lighter sentences than that.
There are also strict rules about publishing in Germany. In this country, no one can get away with saying they're just the publisher and not the author of the information they dissimenate. The editor in chief is legally responsible for the content of print publications, and at radio stations, the program director is liable. Even the publication of a letter to the editor that violates speech laws can lead to a case against an editor in chief -- and it is possible for a publication or media outlet to be shut down.
A Free-For-All Hatefest
But Facebook is a free-for-all, anything goes. You can express your wish that the chancellor be hanged or threaten to kill the children of parliamentarians. You can also disparage dissenters as "ticks," "lice," "trash" or "rubbish" and then muse on how this waste might be disposed of at the dump.
There are pages calling for the refugees in Germany to be sent to the gas chambers. Others incite people to set refugee hostels on fire so that no other outsiders will dare to come to Europe. You can find every conceivable presentation of violence, boastful displays of the swastika and the glorification of the Nazi dictatorship, concentration camps included.
All of this is publically accessible, even by children. In theory, you have to be 13 to register a Facebook account. But no one reviews whether the information in these registrations is actually accurate. And once you're registered, there are no longer any age limits for the content you can access.
A year ago in September, Chancellor Merkel was promised at a business lunch with Facebook CEO Mark Zuckerberg in New York that the company would do something about it. One of the outcomes of such political pressure was that Facebook declared its willingness to participate in a program combatting "hate speech" initiated by Germany's justice minister, Heiko Maas. The task force has met three times over the past 12 months. A week ago Monday, it convened again in Berlin in order to review progress. The event is called "Standing Together Against Hate Crimes on the Net."
But even today, you don't have to scratch far beneath the surface to find illicit content on Facebook. All you have to do is type in the question, "Did Hitler have the Jews gassed?" The search results include pages and entries claiming the Holocaust was invented by the Jews and that Hitler was used as a scapegoat by "Holocaust super liar Simon Wiesenthal."
"Why would a company take action when the worst that can happen is an invitation to the negotiating table?" asks Christoph Lauer, a prominent German Internet activist who spent five years in the Berlin city-state parliament as a member of the Net-focused Pirate Party. "I can't imagine that the Americans would ever come up with the idea of saying: After the incidents at Volkswagen, we have created a task force to find out what action we can take together."
Lauer has had his own run-ins with Facebook. In November 2015, a user posted the following comment under one of the politician's postings: "Big mouth Lauer, worry about the important things like making sure these dirty refugees don't come -- otherwise you guys will be the first to burn." Lawyers refer to this as attempted coercion, which is a crime in Germany.
Because Lauer was a member of state parliament at the time, police opened an investigation. In order to identify the suspect, who had threatened Lauer using an anonymous name, the officials responsible had to contact the Berlin office of Facebook Germany and request the IP address linked to the comment. The IP address can be used to track the computer from which the message was posted.
At first, it looked as though the issue could be quickly resolved. A Facebook employee in Berlin said she could easily track the IP address. But it didn't take long for things to get very complicated. The police official was asked to submit a written request in English. Facebook has its own form for such incidents called a "Law Enforcement Online Request." After the first attempt, in which he detailed the legal basis for his investigation, the officer received a request from Facebook asking for additional information. It included a request for a screenshot of the page in question. When the official submitted the requested image, he received an anonymous message from the Law Enforcement Response Team thanking him for the correspondence and stating that the only option available for further information would be through international requests for judicial assistance. Thank you and goodbye.
For most, the process would end here. Many investigations don't even get that far because Facebook doesn't answer requests for assistance from German authorities or because the forms demanded by Facebook are so well hidden that the police simply give up.
But the episode with Lauer had an epilogue. After officials got lost in the Facebook's murky realms, Lauer got in touch with Eva-Maria Kirschsieper, the company's top lobbyist in Berlin. "Please describe to me the precise steps a police employee has to take in order to obtain the IP address behind a post that contains a death threat," Lauer asked Kirschsieper.
Kirschsieper answered, "I'm very sorry, but I cannot tell you. There are no step-by-step instructions available on what has to be done to obtain an IP address. Each case is reviewed individually."
Lauer then asked in response, "The CEO of your company spoke just a few days ago about programming artificial intelligence to help run his home and you can't tell me what a law enforcement agency has to do in order obtain the IP address of a perpetrator who used your company's technological infrastructure in order to make a death threat against a politician?"
Kirschsieper answered, "Because each case is different and evaluated on an individual basis, there is no standard procedure."
Is Hate Speech Allowed To Drive Traffic?
The game could just keep going on and on -- that is, if you have time for it. Lauer eventually gave up. "If Facebook refuses every form of cooperation even with a relatively well-known person like me, then how are normal people faring?" he asks.
The former Pirate Party member has a theory about why the company has no interest in taking action against members who violate German law. If the perpetrator cannot be identified because Facebook keeps that person's identity secret, then criminal proceedings won't lead to anything. As such, Facebook is offering de facto protection for hate speech. And that hate speech helps grow its traffic. Nothing is as good at keeping people coming as anger. Once people get riled up, they become blind to everything else around them. It's manna from heaven for a company whose business model is getting people to spend as much time as possible on its site.
Facebook itself encourages users to use "counter speech" to combat hate, which also helps to drive traffic. It's an infinite loop: hateful comments are followed up by counter speech, which in turn generates further hate. The Facebook-sponsored "Counter Speech Tour 2016," with several initiatives relating to refugees, recently ended.
Online companies like Facebook enjoy the privilege of being allowed to accept third-party information on their websites, sight unseen. Given the cost and complexity of screening content in advance, lawmakers in Germany have excluded such Internet providers from having to take on the same control and review of content that is required of other media. In return, however, the companies are required to respond promptly when informed about violations. If they do not act in a timely manner, then they can in fact be held liable for the content and required to pay damages. That's what Germany's telecommunications law states, and it also applies to social media -- at least in theory. In practice, however, it depends on where the company has its headquarters.
It's not as if Facebook is not deleting posts. The company can actually be quite ruthless when it comes to removing undesired content from the site. Recently, the company attracted the public's attention a number of times because management blocked individual profiles without even providing those individuals with the reasons why. This recently happened to Leo Fischer, the former editor in chief of Germany's leading humor magazine, Titanic. It also happened to Austrian novelist Stefanie Sargnagel.
From the outside, there's no clearly detectable pattern. The assumption is that Facebook relies on algorithms that react to key terms. That would help explain why the company has also blocked sites campaigning for human rights. The advantage of algorithms is that they don't cost anything to operate once they have been programmed. The disadvantage is that while they can immediately detect a breast or a penis, they are unable to distinguish between a joke and defamation.
Why Isn't The German Government Clamping Down?
Copyright violations are also investigated immediately on Facebook. If Prada discovers its logo on a Facebook page, all it takes is a short letter and the Prada symbol disappears. If Holocaust denial were protected by copyright, the problem would likely quickly disappear from the site. Even in the Silicon Valley, keeping freedom of expression in check takes a backseat to copyright protection.
There are plenty of supervisory authorities. That's not the problem when it comes to why Facebook prefers to pursue its own understanding of the law rather than actual German law. There are 14 state media regulatory authorities in Germany whose job is to ensure adherence to the country's stringent youth protection laws. Those who offer content that have the potential to compromise a child's development can face potentially stiff fines. Pornography and excessively violent content are among those things deemed liable to corrupt the young. But the law also covers anything that is capable "disorienting" adolescents in social-ethical terms. In the worst violations, a company can even be stripped of its license and its content blocked.
Facebook is often in the crosshairs of the State Media Authority of Hamburg/Schleswig-Holstein in northern Germany. Much of the content the agency questions is removed by Facebook. That, says director Thomas Fuchs, is the good news. Given that Facebook refuses to block illegal content entirely, it doesn't take long before the same posting appears somewhere else. That's the bad news. But he says it would be possible, at least in part, for a so-called fingerprint procedure to be implemented of the same type used to address copyright violations. That means that an image or a posting could be marked such that it would be immediately blocked. Facebook, however, has no interest in doing this.
For Fuchs, the problems start with the fact that it often isn't even possible to figure out who is responsible at Facebook. Employees at Facebook Germany, for example, always point out that they aren't actually legally responsible. And in Ireland, where Facebook has its European headquarters, Fuchs says he is never able to establish contact with anyone. "When we send a written complaint, we always get amusing computer-generated letters signed by 'The Facebook Team'," Fuchs says. "They don't provide any way of writing back to them."
Within the Federal Justice Ministry, officials see the very fact that Facebook is taking part in discussions to be a success. "The first task was to make them aware that there was a problem," says Gerd Billen, a high-level ministry official who carried out the negotiations on behalf of Justice Minister Maas. But he hasn't gotten much further than that.
Millions of Complaints
Billen would like to know how many people are employed by Facebook to remove hate speech. Or the criteria under which they operate. Facebook says that the lion's share of the work is done at its European headquarters in Dublin. Last year, the company also contracted with Avarto, a subsidiary of German global media giant Bertelsmann, to assist with additional personnel. But not even Billen has been able to get reliable information on the the precise number of workers who are hired to keep Facebook free of illegal content.
Staff within the Justice Ministry estimate that, at most, there are a dozen people working for the company in Dublin who speak sufficient German to be able to determine whether a post should be considered satire or a violation of the law. The job is also proving to be a strain for Avarto.
It is estimated that Facebook receives a million complaints each day. Complaints could be a person who appears in a photo who doesn't want to be seen on Facebook. Other users complain about Spam or that their favorite football team is being slammed. But filtering out the wasteland of hate speech requires a sufficient number of trained staff -- not just a handful of "customer care agents" whose main qualifications, other than "excellent German skills," are their "flexibility" and "team spirit," as described in a job listing at Avarto.
When you ask officials in the Justice Ministry why it is still possible to read on Facebook today that Hitler never gassed the Jews, a spokesperson says he can't explain it himself. He suggests asking Facebook. If you ask the press office at Facebook, they tell you that the company's employees aren't infallible. They add that the question of whether each individual case is truly Holocaust denial is a difficult legal question.
But Facebook also doesn't take queries from the press very seriously. Johannes Boie, who reports on digital culture for the respected Süddeutsche Zeitung newspaper, recently quoted from his exchanges with Facebook's press office about the company's deletion practices. A spokeswoman for Facebook Germany provides the company's standard reply: "We do not provide details on the number of deleted or blocked postings." On one occassion, she felt it appropriate to answer an entire list of questions with a smiley emoticon, Boie claims. Would it be possible to imagine a company like Volkswagen or Siemens answering with a smiley when asked by a journalist why the company tolerates mass violations of the law?
Facebook's Biggest Critic
Facebook's most dogged critic lives far away from Berlin in Würzburg, a Bavarian city where lawyer Chan-jo Jun has filed legal complaints with prosecutors against all of Facebook's executives -- for incitement, presentations of violence, libel and slander -- the whole works. The list begins with Mark Zuckerberg, continues on to executive Sheryl Sandberg and goes right down to the Hamburg-based head of Facebook for Northern Europe, Martin Ott. Jun obtained his address from an acquaintance on the German business social network Xing who went to university with Ott.
The complaint against Ott hit home. Ott had hardly been officially informed that charges were being considered before attorneys with the law firm White & Case called Jun. They asked if they could discuss the issue with him, and if it was truly necessary to drag the Hamburg-based executive into the matter. Of course we can talk, Jun told them, but only if a colleague took notes of the meeting. Interest on the part of White & Case then evaporated.
Over the past 12 months, Jun has assembled a one-of-a-kind collection of hate postings from Facebook. One example from September 2015: "Close the borders and bratatat until the machine gun burns ... cough."
An example from October: "All I can say is take up arms and shoot down the Muslim sheepfuckers."
Or take this example from January: "It's about time to get the ovens back in operation at Dachau. Then there's at least something you can do something with these a-socials."
Another from June: "Place the Jews with their face to the wall, load your rifles and spread the shit across the wall. #SiegHeil." Some of these statements may be legal in the United States, but they are absolutely prohibited in Germany and can open the door to legal proceedings against any company that dissimenates them.
Jun has also kept meticulous records of the responses he has received from Facebook. It's always the same text, sometimes in German, sometimes in English. "Thank you for taking the time to report something that you feel may violate our Community Standards. Reports like yours are an important part of making Facebook a safe and welcoming environment. We reviewed the Page you reported for displaying hate speech or symbols and found it doesn't violate our Community Standards."
Jun's experience has also been that it's easy to get lost in the hall of mirrors when dealing with Facebook. Even sending a letter informing executives of illegal content can be an adventure. Letters are frequently lost or mail attachments allegedly can't be opened. Jun says the people he is in contact with at Facebook never provide their last names.
Recently he had receipts for three registered letters sent back to him, including one from Menlo Park, where Facebook has its global headquarters. In August he had the next legal announcement of his intent to seek charges sent to Mark Zuckerberg. He at least got a receipt with initials on it, but they are illegible -- was it a Judy or a John? "I'm sure no one there will ever remember having received the registered letter," Jun says with a laugh.
Many business lawsuits stall because of difficulty in determining who is responsible, but Jun has stumbled across a case that shows many parallels. Years ago, Siemens got slapped with a 395 million euro fine by Munich prosecutors for violating its obligatory supervision responsibility. Prosecutors were never able to prove exactly who had been responsible in the case, which involved the payment of bribes to land a contract, but it didn't matter. For prosecutors, it was enough to know that the company was unable to prevent bribery because of its organizational structures. At the time, Siemens' compliance department had six employees. Jun believes that if Facebook only has 15 people in Dublin working to respond to umpteen thousand requests each day, then it's comparable to the situation at Siemens.
Jun didn't get very far in his first attempt to challenge the company. The public prosecutor's office in Hamburg either closed the proceedings or, in the case of Zuckerberg, never even opened them. In the notice of the suspension of the matter, officials noted that Facebook Germany is no more than an advertising sales unit. You can almost see in the letter how pleased Hamburg were to avoid going after the company -- one that, de facto, has no real address in Germany where action can be taken while, at the same time, having access to the world's most expensive lawyers.
Now Jun is attempting to pursue his case in Munich, where he may have better luck. Several days ago, he submitted a new request for charges. Before doing so, he inquired with officials in the Bavarian state Justice Ministry how they perceived his prospects for success. They informed him that they could not provide legal advice on the individual cases. But then an undersecretary answered in remarkable detail. "All we need right now," Jun says he was informed, "is someone who is prepared to initiate proceedings against a company whose executives conceal themselves behind foreign companies."
Changes Could Be Forced
There are a few proposals out there for instituting change. In June, the national body made up of justice ministers from the 16 federal states in Germany launched a legislative initiative to introduce a law which, if passed, would require operators of Internet platforms to immediately disclose the identity of users whose online actions are the subject of criminal proceedings. The law explicitly covers companies that "are not based in Germany, but in fact do business here."
Justice Minister Maas must now introduce the draft law to Chancellor Merkel's cabinet, but he's hesitant out of fear of a backlash among a net community that still views Facebook as a symbol of Internet freedom. So far, he has done little that goes beyond appeals. If he wanted too, however, Maas could push for a further tightening of the country's telecommunications law. All that would be needed is a clause stipulating that every Internet company that does business in Germany would be required to name one person within the firm who is a resident in the country who could be held liable under German law.
Facebook has shown in a number of instances in the past that it can react immediately when shareholders get nervous. Perhaps you don't have to go as far as Christopher Lauer's suggestion that Facebook's network speed be temporarily lowered by legal order as punishment for illegal posts. All that's really necessary is for a public prosecutor to file charges against Facebook somewhere in the country or for a government agency to impose fines in the same way the Americans have recently done to Volkswagen and Deutsche Bank.
The day that happened, company executives in Menlo Park would be forced to send out an "ad hoc" press release to the world. Anything that has the potential to influence the company's stock price has to be immediately reported to shareholders, even charges filed by a small public prosecutor in distant Munich.
The article you are reading originally appeared in German in issue 40/2016 (October 1st, 2016) of DER SPIEGEL.
Click on the links below for more information about DER SPIEGEL's history, how to subscribe or purchase the latest issue of the German-language edition in print or digital form or how to obtain rights to reprint SPIEGEL articles.