Jacob Ward, a 2018-19 CASBS fellow and a technology and science correspondent for NBC News, discusses techno-utopianism and its relation with Silicon Valley.
National Academies Report on Sexual Harassment of Women
"Declaration of Independence of Cyberspace" John Perry Barlow
New Moral Political Economy Project
Federal Reserve report on economic well-being of U.S. Households
Kahneman and Tversky’s (both former CASBS fellows) classic paper on Judgement and decision-making
Center for Advanced Study in the Behavioral Sciences
@casbsstanford on twitter
John Markoff: Welcome to Human Centered, a series of short conversations with researchers at the Center for Advanced Study in the Behavioral Sciences at Stanford University. The center was founded in 1954 to encourage interdisciplinary research focused on the most pressing societal issues of our time. Each year, a range of scholars, scientists, and government officials come to spend a year studying contemporary societal problems. My name is John Markoff. I'm a science and technology writer and former reporter for The New York Times. In these conversations, we've set out to find interesting projects at the center that shed new light on the way we think about society. This week I'm talking with Jacob Ward, who is one of this year's fellows at CASBS. Jacob is in the midst of reporting a book about the effects of artificial intelligence on human decision-making. He has a diverse background in science and technology reporting, ranging from being editor-in-chief of Popular Science magazine to serving as a television correspondent for Al Jazeera. Covering science and technology. This week we spoke about the techno-utopian movement and its relationship to Silicon Valley. So I'll dive right in because we sort of started to have this conversation about techno-utopianism, this idea that technology solves all societal and economic problems.
Jacob Ward: Right.
John Markoff: And I was particularly interested in talking to you more about it because I've been struggling with it with relationship to my character in the biography I'm writing, Stewart Brand, who's been labeled as a techno-utopian in some cases. And my sense about his role and his impact is it's much— it's nuanced.
Jacob Ward: That's right. That's right.
John Markoff: And the first thing I wanted to ask you is, has— I mean, you're writing about AI now in the Valley. Has this come up at all in your project? So maybe you could talk a little bit about your project and—
Jacob Ward: Sure. It's endless. It's endless. So I'll start just with a little bit of background that I, you know, I've struggled with this question of utopianism sort of throughout my career. And, you know, I consider my career to be a small sliver of your career, and so it's fine to be talking careers with you. But I, you know, I was the— at like 20, you know, in my early 20s, I became a reporter for the Industry Standard way back when, and it was the early— for anybody who doesn't know, it was the early business magazine of the internet. Print edition, which is ridiculous, but that's totally what it was. And the— there was a tremendous amount of utopianism going on there, or not so much the utopianism, but it's the thing that I'm sure you've encountered too among business reporters where there's this sort of— it's almost irrelevant to talk about the long-term effects of a thing, irrelevant to talk about the social effects. The thing that we're really interested in is will this company work, will it stay afloat, will it get across the desert. The Valley of Death and get to market, right? Will it get punished by its shareholders? What are the boardroom intrigues? So the kind of the intellectual currency of covering those companies at that time had nothing to do with what I as a cynical 20-something liberal arts major wanted to talk about, which is will this stuff be good for society? Why are black people being excluded from the investment community? You know, why do companies cave to political pressure when they have Planned Parenthood as a client, which is one of the stories I got. So I quit that job a couple years into it at a time when that magazine was flying and they had offered me, you know, you could go to New York, you could go to wherever, why would you leave? And man, I just couldn't do it anymore. I just thought, I just can't cover this with, in this sort of blinkered way. But then fast forward several, you know, to many years later and I'm the editor-in-chief of Popular Science magazine and at Popular Science— You can't escape.
John Markoff: You can't, well, I can't You just can't escape it.
Jacob Ward: But at Popular Science, I had decided, okay, I'm gonna go ahead and embrace the utopianism of this brand because Popular Science had always been a solutions-oriented magazine, and I was struggling at that time to figure out how do we compete with somebody like Wired, one of the, you know, best-resourced magazines in our competitive set. And Wired, I sort of— the story I would tell was that Wired was all about kind of Wired was doing a lot of hand-wringing, but not a lot of investigative work and a lot of hand-wringing, but not a lot of solutions. We're going to be all about solutions. I was constantly drawing on this utopian past of popular science where you had these contributors, Steinbeck, these august figures saying— Steinbeck in the '60s was writing about wanting to go to the bottom of the ocean. Forget going to space, let's go to the ocean. That's sort of as reactionary as anybody got at that time. For me, I had to sort of dress up in a utopian posture. How many years did you do that?
John Markoff: Which years were you at—
Jacob Ward: I was at Popular Science for 7 years in total, and I was the editor-in-chief for, I like to say, 2 years, but it was really about a year and a half.
John Markoff: Up to—
Jacob Ward: I left Popular Science in 2013.
John Markoff: Okay, well, so that's a really important timeframe because now we have this AI debate. So the dot-com era, you know, the collapse 2000, 2001, the resurrection after 2008.
Jacob Ward: Right.
John Markoff: Here we come into the teens. Right. And now there's this AI debate and the subtext is all about technological utopianism.
Jacob Ward: Yeah.
John Markoff: And people are trying to— it's different, I guess is my feeling, than those two periods that you were—
Jacob Ward: That's right.
John Markoff: —so engaged and backed away and then got sucked in. That's right.
Jacob Ward: I like to think that it's different I believe that it's different because we have a deeper and more sophisticated understanding of what technology is doing to human behavior and to the human mind than we used to. I think that we— so that's the subject of my book, is how AI is going to shape human behavior by not just amplifying certain mental habits but in fact creating new mental habits, which we're beginning to see. One thing that I think it used to be that people thought of only tangible industries or industries that dealt with tangible products as being extractive industries, right? That you mine a commodity until it's gone, right? And that the digital world was just dealing with this open-ended, infinite resource. But what we're finding, and this is how I got led to this project, was I did 2 years working on a documentary about human behavior and how we get manipulated based on sort of fundamental human programming that psychology has revealed to us in the last 50 years. What we've learned, what I think psychologists are beginning to figure out, sociologists too, is that companies that deal in our attention and seek to shape our behavior with their products are in fact also extractive, and that we have a finite amount of bandwidth, we have a finite amount of cognitive of horsepower to give to any given task, and if you burn it during the day, you don't have a lot left. And so I think for me, that's a big part of it, is that the quantifying of human attention as a commodity that is now bought and sold has changed the way that we think about these companies.
John Markoff: Well, that's come to the fore in the sort of the aftermath of the 2016 election, particularly in all of these questions raised by the relationship to Facebook and Cambridge Analytica. Sure. And whether this guy Robert Mercer and Bannon actually systematically used these technologies in a new way, right, right, fundamentally new way. And, and so that's playing out on a national stage, and I'm trying to understand sort of how it's changing Silicon Valley.
Jacob Ward: Yeah, that's right, that's right. I mean, I think, you know, one thing that I think— I mean, I think there are many, many things at work, you know. For me, I had a handful of reckonings, you know. One was in When I left Popular Science, I became a television correspondent for Al Jazeera. Al Jazeera is a fabulous place to work because they had no—I mean, as you and I have both experienced, there's a certain amount of benign neglect you get from your masters when you cover the stuff that we cover. At Al Jazeera, they were really only interested in science and technology inasmuch as it could illustrate social inequality and help us understand a little bit more about the social inequality at the heart of the United States and other places. But the— so in 20-whatever it was, 2014 into 2015, as the revelations came out that the NSA had been surveilling people using Silicon Valley's products, you got this incredible sense that, oh wow, this stuff is not as sort of innocent and convenient and just about entertainment as we thought it was. I remember filing a report at that time pointing out that if something like the Nest system, which knows when you're coming and going from your home, knows the patterns of your behavior, I mean, that is a surveillance agent's dream. And if that thing had the NSA logo on it, you would drown that thing in the bathtub, right? You would never put that in your house. But somehow because it's capitalist and it's America, we think, "Ah, it's fine. This is totally cool." So that for me was a big turning point. I remember December 2016, Garry Kasparov was pointing— was one of the first people to start pointing out that the products we make here in the land of the free are being used in other places to undermine our values. And that was such a prescient thing to point out. And I think is, for me, was a really transformative thing that, oh yeah, this stuff, you know, is not innocent and it's not just convenience and entertainment. And now you see all these people, I mean, I have friends, right, who've fallen out of the business of tech for moral reasons because they say to themselves, my God, the thing I was building is being used exactly as I intended it. Exactly. And look what it's doing to all of us.
John Markoff: Well, so how deep into the Valley's culture? I mean, I see that too. I see it in the engineers who are developing AI technologies, but I don't have a good sense of whether that's becoming a fundamentally a fundamental part of Silicon Valley culture. You see it in companies like Google where they, you know, the employees sort of rebel against doing work for the military. But go up to the C-suite and what's going on there, I worry that organizations like the Partnership for AI are, I don't want to say lipstick on a pig, but I don't trust the that we're going to get to this new place simply by having a new set of ethical standards. You still may need regulations. I think that's absolutely right.
Jacob Ward: I think that there is a— I mean, one thing that we've also seen at the same time that we've begun to commoditize human attention and human behavior through these products, I think these companies have also gotten excellent at commoditizing the idea of being, quote unquote, woke companies. You never meet a head of HR anymore. You only ever meet people who are the head of people and culture. You know, every company, it doesn't matter what they're selling, has perfected this line that we are improving the world, we're changing the world, we're here to do well and we're here to do good. We're going to make you money, but you also get the satisfaction of improving this world of ours. And I mean, supposedly 30, 40 years ago, you could— you had your choice as a products company like Procter Gamble of the top 10 business graduate— business school graduates from all the best MBA programs in the country. Today, supposedly you cannot hire those kids. Unless you have a great message about how you're working on climate change or you're working on social justice or whatever the thing's gonna be. So it's become part of the business culture of hiring these kids. Here's the problem though is, you know, so I was at a thing the other night where there was a bunch of sort of mid-level Google, Facebook, other people, and each of them was— could articulate the line that they wouldn't cross you know, one guy was saying, "Oh, I work on a thing that allows employers to know when their employees are at their desks." And so there'll be a certain amount of sort of surveillance there. But I think that, you know, I know that I won't go any further than that, he said, you know. And so everybody's got their own little, their own private social mores that they have, you know, their own ethical lines they've drawn. And they know that selling AI to the Pentagon is on the far side of it because it's a cartoonish example. But there's no coordinated effort within these companies among the rank and file to say, oh, this is actually— the line is closer to where I live and closer to my day-to-day life. Like, that stuff has not been figured out yet. And I think you're right. I mean, I'd be curious to know, because you know better than I do about the C-suite people. Why aren't they— like, what ultimately limits their ability to be moral about this stuff?
John Markoff: Well, I think that the problem is that they've just been too successful. I mean, I look at— I mean, to name names, I look at people like Larry Page and Sergey Brin, who started out— who I knew when they were in the garage. And, you know, they were the children of university professors. They, you know, they had this the courage to have this statement as part of the founding principles of their company, "Don't be evil." Yeah. And then they, you know, they just got trapped in this, in what they created, which was wealth on a scale that we've never seen before. And they were, I mean, you know, I just see them as individuals who were sort of moved by their success in some way that they don't have control over. That's right. And so they may have these values, but they're stuck by what they've done to the world.
Jacob Ward: That's right. I think that's exactly right. I also, I think there's an allergy in the Silicon Valley culture to history and to really almost any study outside of what they're immediately doing. You hear people sort of saying, "Oh, we shouldn't— we don't need to look to the past. We're all about the future. We're here to disrupt things and make them better. Anything that came before needs to be broken in some way," which is why I think it's so wonderful that you and I get to to spend the time at a place like this, right, at CASBS, where, you know, for me, every time I mention some dynamic, you know, at work in Silicon Valley, somebody here says, "Oh yeah, well, you know, here's 50 years of scholarship on that problem that's been sorted out." And I just think to myself, "Wow, man, like, why aren't—" Yeah, why aren't the Larry Pages of the world here asking social scientists, "Okay, well, tell me about legitimacy. Tell me about, you know, quotas. Tell me about," you know, all these efforts that social and political scientists have made to codify human values in the past, and suddenly these companies are realizing they may have to do that and they don't know how. So there hasn't been—
John Markoff: so there hasn't been a real reframing of this techno-utopian impulse is sort of what we come down to. It's still very much alive. There's been no, for example, there's nothing we would call a privacy or a technology Chernobyl in Silicon Valley despite despite the fact that these guys have been pulled in front of Congress. That's right. Nothing on the scale that's going to change things in a, you know, 180-degree way.
Jacob Ward: I think that's right. And it makes you wonder what it would be. I mean, you know, if we were in a more— if we were in a different environment, you know, it'd be pretty easy to say it would be the money, right, that a lawsuit or a, you know, some sort of, I don't know, some other, you know, hit them where they live kind of kind of effort might make the difference, but I don't know how you affect the bottom line of these companies at all. It's almost impossible to imagine an event bad enough that it would actually— you know, Amazon's clearing what, like a— it's some crazy number, it's profit a day.
John Markoff: Who's saying that? I think Google is at about $2 billion a month in profit now.
Jacob Ward: Yeah, right. I mean, so we're not talking— these are not— you can't use the cigarette company strategy to sue them and make them change their policies that way. I was just looking at a— I was just in an environment— sorry, I was just in a meeting yesterday with some of the sort of leading national lights on sexual harassment in academia and people who had been some of the authors of the National Academy's report on sexual harassment. And they were really interesting because they are thinking of the— how to fight this stuff in the same way that I think a lot of people are thinking about it in Silicon Valley. Can you rely on the good intentions of a moral leader, right? Somebody who thinks like Zuckerberg seems to and Larry Page thinks too. I have the, you know, I have the the good— I have good intentions here. It's me, it's Mark. You can trust me. You know, I've got good intentions. I've been with you all this time. You know, they say you can't trust that, and you also can't trust a purely legal regime to come in and correct this stuff because it's— I mean, they didn't say this, but my read is it's too dumb, you know, and it doesn't account for all kinds of subtle stuff that doesn't fall under the technically illegal category. Instead, they talk about you change the structure of the place. And in their case, they were talking about changing things like flatten the relationship between an advisor and an advisee so that the power dynamic is not as strong. Things like that. I don't know that there are any lessons to be carried over, but I know that the argument Silicon Valley seems to make, which is we are utopians and we have everyone's best interests at heart and we're gonna figure this stuff out the way we always have, just trust us. That's not gonna work. And I believe regulation is gonna be necessary in some way, but man, it doesn't make me encouraged when I hear those guys, you know, a senator trying to even understand what Facebook does.
John Markoff: Oh, well, that takes me to this administration. Right. And you have any sense of how this administration will play in respect to Silicon Valley? Will they be Silicon Valley's friends Will they be their foe? You know, we have these techno-utopians still. How is the Trump administration going to respond?
Jacob Ward: You know, I think that the Trump administration is simultaneously so, you know, simultaneously understands that there is some sort of, you know, I think I always try to remember that like there is no such thing as a political Nobody in elected office does anything unless it has some sort of political reason to do it. There's the rightness of the position, but then there's— it's got to earn you votes in some way. Those two things have to come into alignment for people to do the right thing. I think that there is probably some sense that people can sense that technology is not necessarily having the best effect on society, that it is causing us to be more divisive. I think people are sensing that. On the other hand, I think the Trump administration depends so much more than anyone has in the past on these platforms to do what it does, to stoke the emotions that it tries to stoke. And, you know, I mean, by 2020 when we're seeing, you know, the reelection campaign in full swing, you'll see, you know, they're going to be arm in arm with these platforms again. So it's— I don't see them correcting anything anytime soon.
John Markoff: [Speaker:HEFFNER] So I, you know, looking at the arc of these ideas, I mean, this this sort of technutopian debate came up around the dot-com era. It's actually been almost 2 decades in a sense. I mean, you know, I remember sort of being very sympathetic with John Perry Barlow's original manifesto in Wired. You know, that was sort of in some ways emblematic of techno-utopianism. And then, you know, as I saw the cyber stuff happening, I fell away and I began to realize that the internet wasn't a utopian environment, it was simply a mirror of the world and it reflected everything. So I shifted and then when this last election happened, I realized that we had thought that the internet would push democracy to the world and then we realized that the internet was actually a two-way street. Yeah, it comes back, doesn't it? And that was sort of in the mirror sense. What I think we're missing or what hasn't appeared on the stage, my friend Paul Saffo, who's a, you know, a forecaster he likes to call himself, who's been around the Valley as long as I have, likes to say what we need now and what isn't here is a new religion. And, you know, here at CASBS there is this project called the New Moral Political Economy. And, you know, it is this effort which is trying to recapitulate the things that led to neoliberalism. So I, you know, when I put it in that framework, I think, well, maybe that's a good idea. Maybe the academics have something to provide. But, you know, I don't see it coming anywhere else, and so I don't see this fusion of technology and some sort of new ideology, I guess, happening.
Jacob Ward: I guess I would just say, you know, for me, and I'm really curious to hear your opinion on this, you know, I think that when the primary users of the Internet were, and the primary developers of the Internet were largely academics, or at least people from a pretty direct academic environment, then that was the world it was reflecting. And among academics, and I hear this a lot at CASBS, it's the one place that I find myself sort of departing from the people that I hear. People talk about measuring social difference or measuring, you know, a gap in society, a problem like that. In order to fill that gap, in order to solve that problem, that they're gonna— that, you know, you find a disadvantage in order to compensate for it. That's why you study them. And so I think that spirit was there with the Internet. We're gonna use it to fill these gaps in society, to push democracy to places that doesn't have it, to give economic opportunity to people that don't have it. Well, it turns out you don't make as much money that way in the short term as you do doing other things like capitalizing on people's disadvantage and driving wedges between human beings. And as soon as the user base and the developer community within the internet became mostly people who work in that world, who are, you know, professional moneymakers, then you have a whole other, I think, set of guidelines that begin to run it. And, you know, for me I continuously hear people say things like— so like in my book, looking at questions of bias and human error, I'll give talks to companies about that and I'll always have some bright-faced engineer raise a hand and say, "Well, can't we use AI to compensate for these biases, to compensate for these things?" I think to myself, that is— and I say, "Sure, if money weren't part of it, totally." But you don't get to make money compensating for people's disadvantages. You instead make money by capitalizing on it. You know, I remember one of the stats that changed my world and made me— and I try to remember it in every single thing I do— is that in 2016, Pew came out with a study that showed that 49% of Americans could not put together an extra $400 in an emergency in cash, right? They'd have to borrow that money from somebody, the bank, check cashing place, whatever it would be. Typically a grandparent or something, right? But that's a parking ticket, right? That's like you park in the handicap spot because you were trying to drop off your kids, or you lose a tire on the highway, or whatever the thing is, right? That's you suddenly in the red. And for me, I realized, oh my God. We don't need a lot more efficiency in making money in this world. We are doing it. Like, you know, Florida, 50% of Florida right now, I believe, can't make its ends meet. Striking.
John Markoff: What's the number? 3 people, Buffett, Bezos, and Gates. Those 3? Yes, those 3 people have the wealth of the bottom half of the population. Right, right.
Jacob Ward: You know, so we're not compensating for disadvantages. America is not in the business of doing that.
John Markoff: So then you get to this, I mean, techno-utopianism and this discussion about whether these new technologies are actually accelerating inequality. And that's what, I mean, maybe that debate is not enough on the table, certainly not framed in any national way. It's happened in Piketty's discussion, I guess, of capital. But if there's, if these technologies are by displacing labor with capital are going to accelerate inequality. We're just seeing the beginning of this.
Jacob Ward: That's right. There's a lot of smart economists, right, working on these— on questions of like the gig economy and the rest of it. You know, I always try to find— I'm always looking for analogous areas of study that are totally removed from this world because I, for some reason, I find it easier to process them, I'm sure. But I remember one of them, that always stuck with me is Le Corbusier, right, the Swiss architect, the modernist, invented this thing called the City for a Million People, and it was his concept for a world that would house a million people in 4 buildings, 4 250,000-person buildings, and they're beautiful. You look at the thing, they're these amazing— what do they call it, a transepted building? It looks like an X. From above, right, and these towers. And then in the rendering or in the drawing that he did of it, it's endless pasture land everywhere around them. Cows and so forth. And his whole thing was we're gonna centralize people into these high-rises because that's gonna free everyone up to then enjoy nature and be out in the world, right? You look at that thing though now and you go, that's a housing project. I know what that is. That's a housing project. I recognize that. You know, and yeah, it'd be great if they put that in the middle of pastures and forests so we could all enjoy ourselves, but that's not where those things go. You know, Tom Wolfe made this argument in this book, "From Bauhaus to Our House," you know, that the academic ideals of modernists in Europe, when they were carried to the United States, became the basis of housing projects, became the basis of public housing, you know, the worst places to live. And so I think that for me it's this thing of, you know, yeah, I don't know if it's exacerbating inequality. I think it's about to, certainly with the automation of jobs. And the part that I'm worried about, right, in this book is that it's going to exacerbate instincts that have in the past led tough directions for human beings. And that is certainly a problem.
John Markoff: Well, let me ask a last question because it's kind of a methodology— methodology problem that I'm struggling with is in your book, how are you going to weave these things in? I mean, just the practicality of doing it.
Jacob Ward: Why are you bringing me down, man? Come on, I was going to have a nice Wednesday. I was going to go swan around my Stanford office and think I was doing something. Yeah, how am I going to weave all these things together? You know, so I'm trying very hard to find You know, I was originally sort of pinning my work entirely to sort of explaining to people, giving people kind of a crash course in behavioral science as— with kind of Kahneman and Tversky, who— Daniel Kahneman, one of the most famous graduates of— or graduates is not the right word, but one of the most famous fellows here at CASBS. We're looking right now where we're sitting in this library at a wall of books, and his is one of the prominent ones, all by fellows. And so that was going to be the sort of centerpiece. I was going to say, here are the biases that these researchers and others like them have identified, and here's where they map to current AI projects, and you can see them getting worse. But the problem with hanging out with these people here at CASBS, man, is that they just, they can't help but sort of complicate your thinking on these things. And so I've gotten, you know, I have political scientists giving me amazing books on legitimacy and social uncertainty. I've got sexual violence researchers giving me books on how you— on institutional betrayal. I have a primatologist here, Elizabeth Bonnstorf, who's talking to me about whether AI is going to somehow drive our cognition back toward our divorce from primates, back toward their kind of instincts. I mean, oh my God. I love it. I love it. So I am totally overwhelmed at this moment. But I would say that that idea still holds, that there is universal— or I don't know, a million academics will groan out loud when I say this— but that there are some universal tenets of human programming and human behavior that we are just beginning to quantify, and that codebook for the human mind. When you hand it to engineers who then feed it into machine learning systems, which are really just pattern recognition systems, I think we're in some real trouble there. I think that's gonna, it's gonna hand over a user's manual for our brain in a way that we're maybe not ready for.
John Markoff: Well, whenever I find myself in your predicament, which I'm absolutely in for different reasons. Help me. Well, I just tell myself as an author, you know, you control both the vertical and the horizontal. You've said that to me before, but spin that out for me a little bit. Well, I mean, so when I get overwhelmed with stuff, I have to sort of say relentlessly narrow it down to a couple of simple ideas and drive them through. And you can't do everything, but you can give people a narrative. And so if you know your narrative and you stick to it, I think that's something of value.
Jacob Ward: I think that's really good advice. And I know that, I mean, for all of us, I think being overwhelmed as we are by the The pace of change right now, by the things that are being changed, I think controlling the horizontal and the vertical is a very smart idea. Thank you. Thank you.
John Markoff: Thanks for chatting with us today. And for our listeners, take a look at the show notes for this episode for links to some of the topics and organizations we've discussed.