Interview with Alex Pentland Can We Use Big Data to Make Society Better?

In a SPIEGEL interview, American data scientist Alex Pentland discusses how data streams can be used to determine the laws of human interaction. He argues the information can be used to help forge better societies.

Alex Pentland, 62, heads the Human Dynamics Lab at the Media Lab at the Massachusetts Institute of Technology and is considered one of the world's leading data scientists. In his new book, "Social Physics: How Good Ideas Spread -- The Lessons from a New Science," he argues that human communication behaviors follow the rules of mathematics. He says that with the aid of a computer, it is possible to monitor people in ways sufficient to detect these rules. The use of Big Data is proving to be just as important to social scientists as the telescope once was for astronomers.

SPIEGEL recently sat down with Pentland for an extensive interview about his work.

SPIEGEL: Professor Pentland, you do research on the intelligence of groups. Can a bunch of geniuses act stupid when put into a group?

Pentland: Oh, absolutely. That's how I got started on this. We were setting up a laboratory in India. We had a board of directors, some of them were among the most brilliant people in the world, but as a board, they were completely useless.

SPIEGEL: Why was that?

Pentland: There was just too much ego in the room. When one person started talking, he wouldn't stop for half an hour, so very few ideas were actually put on the table.

SPIEGEL: So even a group made up of smart people won't necessarily act intelligently?

Pentland: No. We are usually taught that intelligence is a function of what happens between an individual's two ears, but group interaction is actually at least as important. The way we manage our economy, our incentives -- everything is centered on individuals. But what we think and do is highly dependent of what our peers are doing and thinking.

SPIEGEL: And what is it that determines group intelligence?

Pentland: According to our research there are two decisive factors. Firstly, it is important that everybody explore ideas outside the group and introduce those ideas into the group discussion. Otherwise it's just the same old stuff over and over again.

SPIEGEL: In other words, you should steal ideas wherever you can.

Pentland: Exactly. We are used to emphasizing individual creativity, but we've found that creativity is mostly just the connecting of ideas that already exist. This is the source of innovation.

SPIEGEL: Usually we associate big ideas with brilliant minds -- that the Theory of Relativity came from Einstein, and cubism from Picasso …

Pentland: … but Picasso and Einstein, they were also swimming in a stream of ideas that surrounded them. If Einstein hadn't existed, someone else would have conceived his theory.

SPIEGEL: How about you? You work on what you call "social physics" -- is your work based on your own ideas, or were they stolen from your environment?

Pentland: Well, the basic theme of the book is almost 200 years old. Auguste Comte, the father of sociology, invented the name social physics. As to myself, my talent is that I listen to lots of ideas and I'm skeptical about them. I'm good at finding ones that don't fit with other things. And I'm good at putting them together.

SPIEGEL: How about the famous eureka moments in the history of science? The most fundamental insights happened, according to the scientists' accounts, when they were doing things like hiking, dreaming or driving.

Pentland: But that proves the point perfectly. If it were a matter of reasoning, you would hear stories that start with, "I was drawing the diagrams," or "I was just writing down the equations." But that's not what you hear. What you hear is: "I was sleeping," or "I was taking a shower." Humans have two ways of thinking: the slow way, which is based on rational consideration, and a second, much faster way, of trying to put things together by association. It's the latter that produces aha-moments, when we realize how beautifully things fit together.

SPIEGEL: You spoke of two factors that determine the efficiency of a team. What is the second?

Pentland: Yes. The second factor is whether or not everybody is talking to each other. This basically ensures that everybody is on the same page.

SPIEGEL: Do you need a boss for this? Or is it actually counterproductive to have a group leader?

Pentland: There is a kind of leadership that is actually very effective. It can be very helpful to have someone watching the conversation, and poking in every once and a while to make sure that the pattern of communication is right. Women, by the way, are very good at this, because they make sure nobody dominates the conversation and that everybody contributes.

SPIEGEL: You are describing the role of a moderator rather than a boss. A boss also has to make decisions …

Pentland: … yes, but not by banging on the table and saying, "This is what you have to do!" Good leadership is about arriving at a consensus. The only exception is in emergency situations -- when the enemies are coming over the hill, the traditional model of the powerful leader turns out to be a good thing. SPIEGEL: Do the rules you are describing apply to all sizes of groups?

Pentland: The larger the group, the more difficult it is to get everybody involved. With six people, it's almost a non-issue. It's a lot harder when there are 150. And at some point you reach a kind of natural limit. Everything beyond a mid-size city gets real hard to organize, with our current technology and given human nature.

SPIEGEL: Does this mean that something like the European community is misconceived and incompatible with human nature?

Pentland: I think we don't know how to administer it very well. An average person has about 150 people that they interact with, and this hasn't changed a lot since the Stone Age. Imagine that those 150 people are scattered across all the different cultural groups in the EU, so that the average EU citizen knows people from Romania as well as Portugal or Great Britain …

SPIEGEL: … which is pretty far from the European reality.

Pentland: That's right. The problem is that you are dealing with cultural groups which are more or less isolated one from another. In the United States it is easier, because this country was highly diverse from the beginning. You had Italians, Irishmen, Indians and Africans all mixed together. And it's a highly mobile society, so that you have people going from New York to Kansas, and from Kansas to Seattle.

SPIEGEL: How could you get a similar degree of engagement in Europe?

Pentland: I will tell you a story from Brazil. There they had the problem that all of the country's different states had very different cultures, and because of this, the country was in danger of breaking apart. That's why they changed the conscription rules for young men in the army: Instead of doing their service in their own state they henceforth had to stay elsewhere. So for two years these young men were exposed to other cultures. And when they went back home, it was these ties that held things together.

SPIEGEL: You think we need a common army in Europe?

Pentland: It doesn't have to be an army. You could do it through the university or education systems. Or by creating some sort of Peace Corps made up of unemployed young people. They could help build infrastructure for Europe -- but make sure they do it all over Europe, and not just in their hometown. That way, use the disastrous levels of unemployment among young people to bind Europe into a much more compatible set of cultural norms.

SPIEGEL: So you not only want to use your concept of social physics to describe social interaction, but you also want to engineer communication?

Pentland: Yes. We visit many different companies and organizations and analyze the pattern of communication.

SPIEGEL: And what kind of results do you get?

Pentland: You can often improve communication in very simple ways: moving people down the hall; setting up a lunchroom; the placement of the water cooler or the coffee machine. What doesn't work is having more meetings, because meetings are typically one person broadcasting and everybody else sitting there. That's not communication.

SPIEGEL: Could you give an example?

Pentland: OK, let's take a call center we investigated. Traditionally, they just had a big meeting in the morning, where everybody received their instructions. And then they just worked. And I said: "No, let's give them coffee breaks together so that they can sit around and chat." In the beginning the management was very upset about this, but then they noticed a dramatic improvement in people's ability to work with customers. The employees obviously shared their tacit knowledge during coffee breaks.

SPIEGEL: But maybe they didn't talk about their work at all. Maybe they just enjoyed the opportunity to chat and were happier afterwards.

Pentland: People talk about work all the time -- about how you deal with customers that react poorly and so on. But even when they talk about finding a babysitter, that's a part of their job.

SPIEGEL: Some of your recommendations are much more radical. You propose some very new kinds of incentives for example.

Pentland: That's right. And here, again, the goal is fundamental change: shifting from focusing on the individual to focusing on the social fabric.

SPIEGEL: What do you mean by this?

Pentland: Well, if I want to change your behavior, I could promise you some money. Or I could give the same amount of money to your buddy, but only if you change your behavior. That will force him to talk to you. And if everybody is part of a network of incentives, then everyone is sitting around saying: "Hey guys, we've got to find a way for all of us to change our behavior so that we can all be rewarded."

SPIEGEL: That's a creative idea -- but does it work in the real world?

Pentland: I'll give you an example. DARPA, which is the Defense Research Projects Agency, wanted to do something to celebrate the 40th anniversary of the Internet, and wanted to highlight the power of social media. So they created a contest in which they placed ten red balloons somewhere in the United States, and the challenge was to find the location of all ten of them within one day. Hundreds of teams competed for a reward of $40,000 (€29,000), and all of the teams used economic incentives, except one …

SPIEGEL: … which was, sure enough, your team.

Pentland: Right. We were the only team that did something different. What we said was, "We will give you money if the people you recruit find the balloon." So it became a question of social networks, and everybody said, "Wow, this is sort of fun." So we were able to recruit about 2 million people and quickly found all of the balloons.

The NSA 'Makes Orwell's 1984 World Look Almost Friendly'

SPIEGEL: And this kind of incentive also works in other areas?

Pentland: We used it rather successfully to save energy in Switzerland. Or let's take public health -- many people would like to lose weight, but are unable to do it because they're surrounded by people who don't lose weight either. This works much better within a support group. There are, for example, scales that tweet your weight every day to your workgroup, and if someone's weight goes up, everyone gives them a hard time. And if it goes down, everybody congratulates them. People are scared to do it, but I tell you: It works amazingly.

SPIEGEL: If social incentives work that well, why aren't they used more widely in industry?

Pentland: That's a good question. There's a lot of research showing that economic incentives don't work very well, but politicians and business leaders still always only talk about individual incentives and not about social incentives. They just don't think about people as members of a social fabric.

SPIEGEL: But we all want to be seen as individuals, and not as dependent on what other people do.

Pentland: Well, people feel uncomfortable when you doubt their free will. But as a matter of fact, most of our behavior is stuff we just pick up from each other -- and it's good the way it is. That's what we call culture. Right? Culture is good. Without culture we wouldn't survive.

SPIEGEL: So far we've been talking about social physics without even mentioning Big Data. On the other hand, you claim that Big Data  is fundamental to your field …

Pentland: Yeah, Big Data is to the study of social behavior what the microscope was to the study of bacteria. If you want to construct a better society, you need a complete picture of social interactions. Until very recently we had neither the data nor the mathematics to analyze it. But now, thanks to Big Data, we can know exactly who interacts when, where and with whom.

SPIEGEL: And how do you do this?

Pentland: First of all, you have to write down exactly what you're going to do, and then a federal committee needs to certify that you will protect people's privacy and behave ethically. Finally, you need to have the informed consent of all of the participants to guarantee that they understand what's happening.

SPIEGEL: And what happens once all this is done?

Pentland: In one of our projects, for example, we study young families. First, we give them all brand-new phones with software that anonymously reports who they talk to, where, where they go, what they do on Facebook. I also look at their credit card records and we ask them dozens and dozens of questions. You end up with their pattern of communication across all media, and how this interacts with how they thought and felt.

SPIEGEL: You also read their emails, and examine their phone conversations and bank accounts?

Pentland: No, we don't look at that. We don't record it. We don't look at dollar amounts. We are just interested in patterns: How often do they go to the theater? Do they go to fast-food or high-priced restaurants? How often do they go to the supermarket? And we can see, for example, when people are overspending -- which, by the way, turns out to be a behavior largely driven by social interaction as well.

SPIEGEL: Do you intervene if you see a family with a heavy overspending problem or that a father is drinking too much?

Pentland: No, never. But we might in the future. The more science is moving forward and the better we understand human behavior, the more you get the obligation to act.

SPIEGEL: So by using Big Data you hope to create a better society?

Pentland: That's right. And, in this area, what we need more than anything is a discussion about how we share data -- what I've called a "New Deal on Data". Because the solution to all of our big challenges -- global warming, management of traffic, new epidemics -- depend critically on how we share our data.

SPIEGEL: But this would also increase the risk of abuses.

Pentland: Not if everybody has control of his or her own data.

SPIEGEL: This sounds pretty naive. Taking into account the enormous amount of data circulating about everybody, how do you want to enforce control?

Pentland: This is easier than you might think. Take your bank: You can go online and see your money, right? And nobody knows about it except your bank. And if you don't trust them any more you can close your account at any time. The same principle could be applied to other areas as well. It may sound complicated, but it's not.

SPIEGEL: But with your "New Deal on Data" people would be able to give them permission to share my data with others.

Pentland: Your cell phone provider already knows where you are and who your friends are. The bank knows about your money. Hospitals know about your health. And they want to be able to use these data to provide better service, to increase your wealth, to improve your health. But they can't. Thousands of people are dying because the hospitals can't share their data.

SPIEGEL: But if you allow them to exchange data at will, this might mean trusting them too much.

Pentland: We're talking about highly regulated industries. You can make laws that force them to let you know what information they have about you. There are already such laws. For instance, in Switzerland, all medical information is now kept in a way that is controlled by the patients themselves.

SPIEGEL: And how about the new superpowers in the world of data -- Google and Facebook? Do you want to extend their freedom to exchange data as well?

Pentland: You're right. Right now, it's like the Wild West. But there is hope. Once you can show that the "New Deal on Data" works in some areas, then you can go to the Facebooks and Googles and tell them: "Sorry, but you see, it works." There is no reason why what works for banks, shouldn't work just as well in other areas - except that Facebook won't be happy about it.

SPIEGEL: And what happens when it is the government collecting the data? The recent scandal around the NSA has shown us how eager the secret service is to learn everything about us.

Pentland: That's a huge danger. There is nothing more dangerous in this new world than somebody with a lot of big computers and the ability to collect a lot of data -- whether it's any company or the government. I think, the big mistake of NSA was to have such a centralized database. They make Orwell's 1984 world look almost friendly.

SPIEGEL: And it raises the danger of leaks. Luckily it was only Snowden stealing …

Pentland: Well, do you think so? Snowden just happened to tell everybody about it. I'll bet you can find five other people who stole the same data and sold them rather than telling us about it. That's the real stupidity of NSA.

SPIEGEL: Professor Pentland, we thank you for this interview.