How can scholarly researchers and government policymakers advance their collaborative relationships in service of generating evidence-informed outcomes that yield more prosperous, equitable, and inclusive communities? Panelists Jake Bowers, Carrie Cihak, Dan Hopkins, and Piyush Tantia join IDinsight CEO & 2019-20 CASBS fellow Ruth Levine in this enriching conversation.
Causal Inference for Social Impact Lab
Narrator: From the Center for Advanced Study in the Behavioral Sciences at Stanford University, this is Human Centered. The evidence-based policymaking movement has grown substantially over the past 25 years. Government officials, researchers, and the public are interested in ensuring that policies result in intended outcomes while being an effective use of public and private investments. Since the Foundations for Evidence-Based Policymaking Act of 2018, we've seen the creation of more and increasingly diverse types of administrative data, greater integration across datasets, and more governments using data in their programs. Notwithstanding this progress, there remain practical, methodological, and economic limitations to evidence-informed policymaking that must be addressed. Today on Human-Centered, another episode from our Social Science for a World in Crisis series. This episode, which originally webcast Thursday, November 10th, 2022, is titled Improving Academic-Government Collaboration In Evidence-Based Policymaking. And it includes panelists Jake Bowers, Associate Professor of Political Science at the University of Illinois Urbana-Champaign, Fellow in the Office of Evaluation Sciences at the U.S. General Services Administration, a 2018 CASBS Fellow and a current CASBS Research Affiliate. Carrie Cihak, an Evidence and Impact Officer for King County, Washington. She was a CASBS Fellow in 2017 and also a current CASBS Research Affiliate. Daniel Hopkins, professor of political science at the University of Pennsylvania, member of the social and behavioral science team during the Obama administration, and works currently with the city of Philadelphia to improve policy and implementation. And lastly, Piyush Tantia, chief innovation officer at Ideas42, a social enterprise that uses insights from behavioral science to improve lives through better policy and systems design, and also a 2021 CASBS fellow. Moderating the panel is Ruth Levine, CEO of IDinsight, an analytics and research firm that helps global development leaders design and test policies to support communities. She's a former lead of the Global Development and Population Program at the William and Flora Hewlett Foundation and a 2019 CASBS fellow. Let's listen in as they explore ways in which scholarly researchers and government policymakers can advance their collaborative relationships in service of generating evidence-informed programs that yield more prosperous, equitable, and inclusive communities.
Ruth Levine: Hello, I'm Ruth Levine, CEO of IDinsight and true believer in the power of data, evidence, and reason to help us all live better in a messy world. I'll be the moderator for today's session. Welcome to the 22nd episode of CASBS's webcast series, Social Science for a World in Crisis. I want to acknowledge and thank the partners for this episode: Evidence in Government and Politics, also known as EGAP, Ideas42, Sage Publishing, and my own organization, IDinsight. This event is associated with CASBS's Causal Inference for Social Impact Lab, or CISIL, and we'll provide a link to the project in the chat box. You've probably read the bios of our wonderful panelists in the event promo, but we'll provide a link to them in the chat box as well. And I'm going to quickly introduce them before we jump into a conversation. Jake Bowers is Associate Professor of Political Science at the University of Illinois Urbana-Champaign and a 2018-2019 CASBS Fellow. He's a fellow in the US Office of Evaluation Sciences, has served as, as Methods Director for EGAP, and co-founded Research for Impact, which connects academics with practitioners. A perfect participant in today's session. Carrie Cihak leads evidence-informed practice and partnerships for King County, where Seattle is the, the largest part of King County, but not the only part of King County, as we'll hear. Carrie guides county agencies on community outcomes and impact with a strong focus on advancing racial equity. Carrie is a current research fellow and former fellow at CASBS, and with Jake leads CASBS's CSIL. Dan Hopkins is a professor of political science at the University of Pennsylvania. He was a member of the Obama administration's social and behavioral sciences team and has worked with the city of Philadelphia to identify ways that behavioral science can improve policy and policy implementation. And last but not least is Piyush Tantia, chief innovation officer at Ideas42 and a 2021-22 CASBS fellow. He was founding executive director of Ideas42, which is an organization, as many know, that uses insights from behavioral science to find solutions to tough social problems. So here's how we'll proceed. We'll do a few rounds of questions to the panel, and I'm looking forward to hearing from a range of different perspectives and particularly hearing how the question of academic policymaker and practitioner partnerships play out differently at the municipal, county, state, federal, and international levels. I know from earlier conversations that there's a diversity of views on the panel, and so I'm really looking forward to a lively and informative conversation. I'll have the pleasure of asking a few questions and hearing from each of the panelists. And then we'll be able to take a few questions from the audience. So please feel free to submit questions using Zoom's Q&A feature. We'll ask that you keep the questions concise and on point. Given the volume of questions that might come in, we'll select a subset, but then pass the other ones on to the panelists, and they'll be able to respond after the session. So we'll take a few questions from the audience and then close with a kind of lightning round to hear final thoughts from our panelists.
Carrie Cihak: Let's jump in.
Ruth Levine: Okay, friends, I want to be inspired by you. I want to hear from each of you about how you define success in this complex arena of evidence to policy and practice. And one quick example of where you've seen or been part of a successful experience. So, Jake, let me start with you.
Jake Bowers: Sure. That's a great question. Quickly, I also have to say that although I'm a part-time federal employee, today I'm speaking as a professor and nothing that I say comes from the US federal government. About success, I see a lot of success. And in fact, I think I see success every time a policymaker decides to think about policy change as presenting an opportunity for learning rather than like as a risk of failure. I think this growth mindset helps policy improve more quickly and more broadly across jurisdictions. An example of this is, um, are all of the null results that we find that we, that we make public. Every time we present a null result and we understand it as an opportunity for change, for brainstorming, for new theory, we're sort of understanding what we're what we're doing as learning to improve rather than as risking failure and hiding, sweeping things under the rug. The example I'd say is I'm really proud of the US Office of Evaluation Sciences and 50 null results, or the null results that my collaborators at the Policy Lab at Brown University with the Rhode Island Department of Health published in Nature last year. So it's funny, you might not I don't think I'd say null results and growth mindset as success, but I really find it exhilarating and successful.
Ruth Levine: Yeah, great. Thanks for that. And that broadens our universe of successes significantly if we include null results. Carrie, over to you. What's success to you and what have you been part of?
Carrie Cihak: Yeah, well, I definitely agree with Jake. One of the studies we learned a lot from was a study that had a null result. But when, when the term success, when I think about that, what I think about is when I see an aha moment light up in a colleague's eyes. And that could be, you know, from understanding— they suddenly understand the difference between an output and an outcome, or they hear the result of an impact study of a program that they've been working on for, you know, 2, 3, 4 years. Or a lot of times having our staff recognize that they themselves are researchers too. I see those aha moments. And sometimes I've had the fortune of participating in kind of a community-wide aha moment. About 7 years ago, King County voters chose to tax themselves to launch an initiative we call Best Starts for Kids, which is our comprehensive early childhood development initiative. And when we built Best Starts for Kids, we took a somewhat different approach than we usually do when we're creating like big big packages, big initiatives, which is to really focus on the outcomes, not the programmatic— individual programmatic inventions, to ground it in science and evidence, and to have it really be driven by equity. And that outcome, evidence, and community-based approach helped us build a really strong coalition that was full of aha moments, which was really wonderful. And I'm really happy to report that after 6 years of implementing Best Starts for Kids, King County voters overwhelmingly renewed it at double its level of investment, $800 million, and using evidence to ensure that every child in King County reaches adulthood healthy, happy, safe, and thriving. And that initiative is now, you know, influencing the way we do other things in King County too. So success builds on success.
Ruth Levine: Wow, thanks for that. You know, I literally got chills when you were talking about that. That is really, really Great to hear. Thank you for that moment of real inspiration. Dan, you have operated at the federal level. I have also worked in the U.S. federal government, and I know that finding success can be a challenge. So over to you for your perspective.
Dan Hopkins: Absolutely. And I'm grateful to be here with such a distinguished panel participating in this conversation. I would say that over the course of my work with behavioral science in government, I have broadened my notion of success considerably. So in 2015, when I first started doing this work as part of the White House Social and Behavioral Sciences team, I had a relatively narrow view of success as a completed experiment or another empirical trial that led to a change in policy, a change in how we implemented, um, some government program or business. But as I have done this work longer, I've realized— and this builds on things that Carrie and Jake have already said— that success is not just changing a policy. And indeed, sometimes you want to know that the policy that you already have implemented is working well. Success can often be found in the process, in the learning, in identifying what kinds of outcomes would even let you know if your policy is, is doing what it's supposed to. And so the very process of sitting down with academics and citizens and policymakers bringing them together and starting to try to define what is, you know, what is a good outcome? How do we measure it? What are the interventions that we have control over? What are the touchpoints that we as a city or as a federal government can potentially influence? I have a much broader notion of success, and partly that's valuable given that my initial narrow notion of success, as Carrie and Jake's comments have indicated, might have run up against a lot of results that didn't decisively say that one policy course was better than another. So with respect to some experiences that I have had, the partnerships that we have built here in the city of Philadelphia have really enabled us to respond very, very quickly to some of the most pressing issues here in the city. So within a month of the onset of COVID-19, we were surveying people throughout Philadelphia to try to learn and identify some of the key barriers to keeping our neighbors, our citizens safe. And so I was very proud that our existing collaboration meant that at a moment of real crisis here in the city, our colleagues in the city government didn't need to figure out who to call. We already had an existing relationship and we already had a network and a survey that we could leverage. We also got to work very, very quickly, and I hope effectively, when COVID-19 changed how many people were voting. Philadelphia is the largest jurisdiction here in Pennsylvania, itself, you know, one of the perennial swing states. And so it was really important that all of our citizens were able to cast meaningful ballots in the 2020 and the 2021 elections. And so we did an ongoing set of projects to reduce the rate of ballots that were cast, so-called naked ballots, where people were casting ballots outside of a secrecy envelope. We did extensive work to try to leverage— build people's relationship with the city and leverage that into increased voter turnout. And so we're very proud of all that work.
Ruth Levine: Wow, really, really interesting. And, you know, your point about the pre-existing relationships really facilitating The opportunities for researchers and evaluators to help support the COVID response really resonates. That was certainly true with IDinsight. The teams that had these existing relationships with either NGOs in the health sector or with governments really were able to step in very, very quickly. So very, very helpful to hear that, and congratulations on the on the work you did to advance voting as well. Probably had some resonance and spillover to Tuesday, this past Tuesday. Piyush, let's hear from you. Would love to hear how you conceptualize success and what you might think of in your own work when you think about success.
Piyush Tantia: Yeah, no, thank you. Great to be here. I'm going to build on what everyone's been saying and really on this relationship point. So we just we released our second impact report. We've been in existence about 14 years, and I was struck by some of the numbers. So we total up, you know, what we've done over the last several years. We've worked with 288 different partners across 55 countries, and that's amazing. I mean, that's a lot of relationships and a lot of learning and a lot of people accepting this way of, of doing things and using evidence. So That's heartening. We've directly impacted 30 million people just from the projects that we've done, the experiments we've run. If those are scaled, right, that reaches an order of magnitude more. And now what we're starting to do is scale up things that work in one place to other places, and that always needs adaptation. So it's not, not been as easy as just a plug and play. So one of those programs, You know, I love this one. It works on cash transfers in Africa. Around the world, 1 billion people receive just cash assistance from governments. And we've added little behavioral, you know, spiffs to the programs where people set a goal for what they want to use the money for, and then they put aside some money in a burlap sack on the spot because they may not have access to a bank account. And these things, this is now scaling across 10 different countries. And the first one that we worked Madagascar, we saw results like people actually serving more diverse meals and language learning impacts on kids. It's just amazing. So it's really great to be able to see some of these long-term effects and then be able to take it to other places.
Ruth Levine: Yeah, yeah, fantastic. Thanks for that. I just want to highlight, you know, across these comments about how success is defined. I think, you know, a few kind of themes have come out: the growth mindset, the stimulating curiosity, aha moments. I love what you said, Carrie, about having the, the team in government feel like researchers, be researchers themselves, inclusive processes to bring in citizens, and, um, and then the the relationships that I think everybody has talked about. So really a nice broad kind of conceptualization of success, getting away from maybe that narrow and linear, do an experiment, have it taken up, have it moved to scale. So let's, let's turn from the success frame a little bit to the challenges. Because of course there are some, and we're all working hard on them. So I'd like to hear a little bit more about your experience with the interface and interactions between researchers and policymakers, or researchers and practitioners. What are the challenges you've seen or experienced, and how have you found that those challenges can be overcome, either, you know, in the early project design, implementation, or evaluation and feedback moments? Piyush, maybe we could start with you. I'm sure you have encountered a few challenges, as have we all.
Piyush Tantia: Oh yeah, yeah, too many to count since the beginning, but we've learned a lot also. You know, and this— so this challenge still comes up sometimes when an academic reaches out to us and say, hey, can I, you know, partner with you? And usually the ask is of the following format: well, I have this very specific idea for very specific intervention that I want to test, and this is how I want to test it. Can you find me someone who's willing to do that? When, when we first started Ideas42, that's what we were doing also. We, we would sit in a room, we would generate ideas, and then we would go out and try to find somebody who was willing to, you know, try that thing out. We quickly realized that while we were able to find people to do this, they would often back out somewhere along the line, or that project wouldn't work out because it wasn't a priority, right? They were just doing us a favor. They were just testing that thing because we asked, or famous academic came along and asked. So now, uh, what we do is the opposite. We start with the partners and the communities we're working with, and we say, okay, what are you really trying to solve? What is really top of mind for you? And where you know, which of those issues could we apply social science methods to and leverage the research that's coming out of academia? Then we matchmake the two sides and those projects work out much, much, much better. Some of them have led to, you know, publications in top journals when we've done a three-way partnership with an academic, Ideas42, and the implementing partners, which are often often governments, but not always governments.
Ruth Levine: Yeah, yeah, that kind of moving in a sense from supply side of ideas to demand side for solutions and for support for decision-making is a key move and not easy to accomplish. Jake, what are your thoughts about this question? What challenges have you found?
Jake Bowers: Yeah, you know, Piusz really kind of said a lot of what I was also thinking. I mean, you know, we've seen projects not go forward because you can't share data, because you can't look at the data, the data quality. You know, there should be an ID number, but it has the word dog in it. You know, there's unbelievable challenges that we're constantly facing. But actually, trust was what came to my mind as well. I mean, trust in each other as human beings involved in a collaboration. Also, you know, the idea— trust that the idea is that the results will be useful no matter what. Kind of coming back to my first, you know, idea of growth mindset, which I know was developed for education, but really seems to work here that, you know, we're doing this research not in order to rank you and to say that you're doing a good job or a bad job, right? The research will be helpful regardless. And in fact, the process of the research, as, you know, Dan was suggesting, and actually everybody here has suggested, is very useful. However, we've seen trust fail in many ways, especially across sectors. There are lots of reasons why people have stereotypes of each other, right? The, you know, classically, you know, the policymaker is really prepared to be spoken down to and to interpret almost anything as if you're being spoken down to. The academic is really prepared to hear ideology or, you know, from the person who's in the government, you know, to enter a relationship prepared to hear negative asking things from somebody else is a bad start. So, you know, how do we overcome it? I think institutions help, like, you know, places like Ideas42, the OES, King County, what Dan is doing. All these places help train people on both sides how to not begin relationships with these possible negative starts and how to name and talk about incentives, you know. Junior faculty members are desperate to not be fired. They need to publish stuff. It's super important to them, and they need to— and their collaborator has a heart and wants to help them, right? And their collaborator's really worried about being embarrassed by publication of things. And so how— but the academic also has a heart. So how to figure out how to kind of create these trusting relationships has— that I think has led to successes by these organizations because one-on-one you can just be lucky, you know, that two people get along and that two organizations can kind of get along. But I think that, you know, these— there's growing knowledge about this. Like the Research for Impact group is already working on this. Adam Levine, my collaborator, is writing a book on it. So, and there's institutionalized knowledge in Ideas42 and all these other places that we're talking about. So that's hopeful. But anyway, that was my response.
Ruth Levine: Yeah, yeah, yeah. Really interesting. And I appreciate you sort of highlighting the maybe incentives that are in tension with publishing and maybe not being quite prepared to be fully transparent about what's working and what's not in the public sector, because as we know, that can put a target on the back of either an elected or an appointed official. Carrie, you're in government. You worked with a lot of academics. Tell us what— tell us what the challenges are.
Carrie Cihak: Yeah. So I think, you know, a very practical challenge is timelines. Often researchers are working on timelines that are really out of sync with decision-making that has to happen in particularly local government. And researchers also want to be really confident about their results. So take the time for peer review and really understand their analysis. And I totally appreciate that. At the same time, I think it's also helpful for academic researchers to understand that policymakers are making decisions all the time with a huge degree of uncertainty. We're really comfortable with uncertainty. And so often understanding the results of a study that's in progress sooner is better than getting a really confident result later. And for policymakers, I think we really have to approach, you know, as we've heard from Jake and Dan and Piyush, we really have to think about evidence building as continuous learning and understand that as more data and analysis occurs and our learning deepens, what we know and how confidently we know it, that naturally changes. And that was really evident, you know, in our learning and our response around COVID-19, for example. And that's then often subject to a lot of criticism. But I really believe that rather than using that, that change in what we know and that learning, rather than using that to dismiss the science, we should use it to really celebrate our capacity as humans to learn and to adapt. I also think there, there are, you know, challenges around objectives. Often, you know, policymakers aren't really clear about the question we're asking, and so the answer we get back doesn't match with the decision we're having to make. So we've got to be really clear about, you know, our timelines, our constraints, outcomes we really care about. So, for example, we might be more interested in not the average effect, but on the effect of an intervention for a particular population. If we're not clear about that at the outset and engage researchers from the outset, it's really hard to kind of back engineer to get to those, those results.
Ruth Levine: Yeah, very, very helpful. Could you say a couple of words about whether you've experienced the kind of tension around the push to publish?
Carrie Cihak: Yeah, absolutely. I mean, I think that's inherent in so much of our relationships with academic researchers. What we try to do is really create forums within government where we can explore what researchers are finding and what's emerging as they're learning. So we're co-learning together and then I think that's also really helpful to researchers because we're bringing our knowledge about the context to bear on the kinds of things that they're looking at, right? So, for example, you know, if, if you had a study that showed that offering people a free fare didn't result in them riding transit more, does that mean that Does that, does that mean that a free fare is not impacting people's mobility? Or does it mean that we aren't organizing transit in a way that takes people to where they need to go? Those require really two different interpretations and completely different policy responses, right? So it's, it's not just that numeric result that's so important. It's really working together to unpack what that means. And ideally, we want to kind of have that richness of that context in a publication too, so that it informs not just what's happening in our jurisdiction, but what's happening elsewhere.
Ruth Levine: Yeah, yeah. Thanks for that really, really interesting example. Dan, it sounds like you've had a very deep engagement with the City of Philadelphia and undoubtedly other kinds of partners. What are some of the constraints you've encountered, and if you've overcome them, and how you've overcome them. Would love to hear about that too.
Dan Hopkins: Absolutely. Here in Philadelphia, you know, I live and work in Philadelphia. Philadelphia is the 6th largest city in the country, but there is no city that's bigger and poorer than Philadelphia. We face high levels of poverty, and that colors both the resources that we have and some of the central challenges that we face. It's also not surprising since I'm here in Philadelphia that I'm gonna use a metaphor related to eagles. So when baby eagles are born, they spend about, you know, 9 or 10 weeks in the nest, and then they make their first flight. And only, unfortunately, about half of them actually successfully make the first flight. The other half don't. And I offer that metaphor kind of building on what Piyush had said earlier, because many of the projects that— the collaborative projects that start off don't make that— don't make a successful flight, right? For various reasons, they may not be successful at the end. And from an academic side, if you really need this one study, that can be a challenge. I think one of the advantages to being a more senior researcher is that I can, you know, I can afford to have several projects that don't necessarily go where I anticipate or don't wind up with publication. So I certainly think that academics going into this kind of work have to take a long view, have to take a relationship-building view, and a very client-centered view as well to learn what are the questions that the city of Philadelphia needs to answer. I also think that there is a real challenge in that some of the agencies that collect the best data and are the most straightforward to work with may also not be the agencies that have— whose questions are the most pressing or in some ways the most in demand from a justice vantage point. When I worked with the federal government, I did a lot of work with the Department of Interior, on American Indian-facing policies. And one of the challenges is that I could approach with an idea about how to send text messages to increase attendance at Bureau of Indian Affairs or Bureau of Indian Education schools. But if the infrastructure isn't there to send text messages, you know, that idea is going to fall flat. And I have found much more generally that many of the agencies that are hard at work trying to reduce the burdens of poverty, work with populations like American Indians, African Americans that face systematic and historic disadvantages, is that those agencies are politically under the gun. They are very worried about evidence being used against them. And they also are dealing with a legacy of extensive, long-term underfunding. So oftentimes their data collection apparatus just isn't as developed. And so I think that one of the challenges that researchers really need to keep in mind is that the natural process of finding partners who identify outcomes and who can produce a zip file with all the data that you need may sometimes lead us away from the problems that are most pressing from a public policy point of view.
Carrie Cihak: Thank you.
Ruth Levine: Gosh, that is such a valuable point. Yeah, such a valuable point that the government agencies that we might be able to make the biggest contribution to in concept may have the most complicated kind of enabling or disabling conditions at the outset. Really, really valuable insight. CASBS, thanks for that. So, you know, there are different models. You've spoken to some extent about some of them. There are different models for an interaction or an engagement between academics or other researchers and people in the policymaking and practitioner policy implementation world. So, you know, we've seen a rise maybe a fall as well, of kind of evidence units within government and embedding a research team in government. There are some great examples and some cautionary tales, I think, there. There are organizations like Ideas42 or J-PAL or IPA that do some version of matchmaking between partners. There are a range of different models, so I'd love to hear any reflections about from your experience or what you've heard from others, what you think the kind of models of partnership are that are most effective. And I know you've raised this a little bit in your remarks so far, but I'd love to hear a little bit more specificity. So let's see, who should I start with here? Dan, maybe you had the last word there, but you were— you stimulated this question. So if you want to go on a little bit more about the models that you've seen work well.
Dan Hopkins: Absolutely. I think that effective models emerge from mutual understanding to an important extent. I think that in addition, the— one of the things that I have realized, and I really liked that Carrie had given voice to the idea that many of our public servants are also researchers at the same time. And I have found consistently that public servants who work hard on various policies are very curious about the impacts of their policies, very open to learning more about them. So I think that effective partnerships are going to be long-term. I think that they are going to be— I also, I think that there are a lot of win-wins. In the sense that for academics, ultimately, publications, knowledge generation, um, that's one of the goals. But academics so often have interesting ideas and they want to know how they work in the real world, and instead they are often for, you know, they subject a group of 18 to 23-year-olds to those ideas. Maybe if they're lucky, they'll put them online and get quite random people who are up late at night on Amazon's Mechanical Turk interface willing to do our experiments. And so for academics, the opportunity to see whether these ideas that we've been thinking about and working on for years, whether they work in a real-world context, and since so many ideas work in one context but not another, which are the contexts in which they work, I think that's a real win. At the same time, particularly for a resource-constrained city like the city of Philadelphia, I mean, we have great universities here from, you know, St. Joe's Temple, Drexel, and there's so much knowledge in the city that the city can leverage for free because academics want to serve the city that they live in. They want to bring this knowledge to bear for the public good. And so I think that, um, you know, modern life, very, very fragmented, very, very specialized. And one of the things that I've really appreciated about these partnerships is that we can build long-term mutual understanding, um, and that is then productive not just of of specific completed studies, but of a culture of learning that goes both ways.
Carrie Cihak: Yeah, Ruth, I might jump in here on Dan's point, if you'll let me.
Piyush Tantia: Sure.
Carrie Cihak: You know, the answer to his last question also stimulated this for me, and I'll just kind of build on what he's saying about bringing, you know, the program staff and researchers together as co-researchers. A third piece of that is really the community themselves, right? And so when I think about building a successful partnership on a, on an evidence-building project, I really like to think about those, those three different, three different entities. So the, the researcher themselves, the outside, that outside expertise. Then we have our program staff who bring a lot of expertise, but community themselves is bringing critical expertise into this. This conversation. And that lived experience, being closest to the problem and helping co-create the solutions, understanding what's going to work, is really invaluable. And, you know, we try to set up a lot of our work so that we've got all three of those kinds of expertise at the table. And we also really want to make sure that everybody around the table really has equal respect for each other in that conversation. And if that respect around the table isn't there, the research project probably isn't going to work out. So we do spend a lot of time trying to, trying to build that, that trust.
Ruth Levine: Thank you very much for that, Carrie, and for jumping in. And, you know, I, I am putting together a couple of points in my head from these last two interventions. And Dan, I take the point that there's a real opportunity for particularly municipalities and local jurisdictions to make— develop good relationships with the academic institutions in their midst. I think it can be a hurdle because those academic institutions don't always have fantastic relationships with the communities around them. So I lived in Baltimore for 20 years. I now live in right near Palo Alto. So without naming any names of academic institutions, I can tell you that, you know, my observation is that there's a pretty big divide between the reputation of the universities and their— the way many in the community kind of feel about whether they have good intentions for the development of the jurisdiction. So I think that, Carrie, your point about the importance of deep engagement with the community really like raises the bar on what university-based researchers have to do to make sure that they're really seen as, as the best actors in the, in the environment. Um, Jake, any, any thoughts about this?
Piyush Tantia: This—
Jake Bowers: sure, yeah, I do. I mean, I think that actually it's really good to talk about models because I think we need diverse models. I mean, we have like the network matchmaking model, right? That's, you know, we've discussed that's super powerful and really effective all around the world where you have individual academics connected through an institution that allows for some continuity. We have embedded in, like, you know, inside the federal, US federal government type of institutions where I can access data that other people cannot access because I have my, like, ID card, you know, here. We also have models where a university will decide, the leadership of a university will decide to try to make a connection. So Brown University, together with the governor of Rhode Island, created a policy lab and had the chief of police of Providence attend the policy lab meetings in order to make that happen, to bring, have to, and we sat in the university, of course we sat on Zoom, but the idea there was, was a university decided to do it. My university is working on its own models. Carrie has created her own model, as far as I can tell, that I kind of— if I wouldn't mind asking her to describe just a little bit about how— is that okay, Ruth?
Ruth Levine: Sure.
Jake Bowers: But those are the models that I've seen at work. And Carrie, can you tell us just a little bit about your, your version?
Carrie Cihak: Yeah, well, I'm not quite sure which version you have in mind, Jake, because I think all these models are so great and we try to execute on all them, right? So we have like, you know, we in our Department of Community and Human Services, we have it and in public health and now growing in Metro Transit, we have people who are, you know, PhD-trained researchers and they're fantastic. And we can, you know, they really have all of that context and that, that knowledge. But I think what you're referring to is simply You know, King County doing a lot of outreach, being— trying to be really clear about what questions are our priorities to engage researchers on. And then we've managed to develop some really long-term relationships with a number of different entities around the country. So the Lab for Economic Opportunities at Notre Dame, the Reg Lab at Stanford, We're working with University of California, Irvine. We've actually got some really great connections at UPenn Medical School. And Dan, let's figure out how we can work together too. And, you know, I— the point was made before, I can't quite remember who was talking about it, but it's that long-term relationship that's so important. So, for example, during COVID we were able to really, you know, work with researchers very quickly in the midst of COVID to kind of understand some of the impacts that we were trying to unpack, which was super, super helpful.
Jake Bowers: Right. What I saw is that you guys have, instead of having a dedicated team dedicated to this, to evidence, you have embedded connections to evidence throughout the whole government caring about these relationships. So it's like a network and without necessarily needing like office space that's dedicated to this. You achieve. So anyway, it's a cool, it's a cool, it's a cool model that I've enjoyed watching.
Carrie Cihak: It's, yeah, it's decentralized.
Jake Bowers: Yeah, decentralized, right?
Dan Hopkins: Yeah.
Ruth Levine: All right, we're gonna have to work on cloning Carrie. I can, I can tell that that's the, that's the route to success here. Piyush, you, you have an organization, you founded an organization that's really set up to be this connecting point. Um, yeah, would love to hear a little bit more about that model and what you've learned over the years. You already cited a few things, but would love to hear more.
Piyush Tantia: Yeah, I want to pick up on a lot of threads that we've talked about, which is, you know, building long-term relationships, the null results being useful. So all of this, what it leads to, is a feature of, I think, across many models that make them successful, which is flexibility and expectations that not everything will work and that you need to iterate. This is a iterative process. You work over years, you try lots of different things, a few things work, right? So portfolio approach. So when we've had our partners, our donors, funders, everyone get that and support that type of model where we can run a portfolio like that, uh, then we see the greatest number of successes coming out of that actually, and the best relationships working. So, our behavioral design teams that we embedded in, in New York City and in the federal government during the Obama White House. I mean, all of those were funded with no specific, like, oh, you have to work on a project on healthcare or this or that. There were no silos. It was just pretty flexible. So I think that's pretty important. And then I'll use that to pivot to a general point about funding that's always bothered me about in this space is somehow there's a lot less funding in this space doing R&D than there is in, say, the physical sciences. But it costs just as much to do these things and try social science interventions in the field and get them right. I mean, billions went into creating the COVID vaccine, but I think the total funding for figuring out how to get people to take it must have been in the tens of millions, maybe. It may even have been only in the single-digit millions. And that's kind of shocking.
Ruth Levine: Absolutely. I think there are a lot of nods around the room, the virtual room. So let's turn— you raised, Piyush, the question of how this work gets funded, whether it's done in a flexible or a more focused way. And by whom and in what amount. So I used to work at the Hewlett Foundation, which supported— continues to support a lot of social science research across various programmatic areas. And I think, you know, always is— all foundations are constantly learning and seeking advice about how to do better. So would love to hear from everybody one by one about any advice you have for funders, including both philanthropic and also through public procurements. Jake, any thoughts about what the funding models should look like?
Jake Bowers: Somewhat. I mean, I was thinking more about, I mean, back to my original ideas about how I'd like to see more null results reported. I'd like to see more press releases like, the Bowers Foundation found that something didn't work as well as we thought it would work, but we're going to convene a bunch of people to discuss why, and we're going to come up with a new study soon. So stay tuned to our press releases. I don't see enough of that, I think, from any foundation. I see a lot more, you know, this, watch, look at the impact we're having, you know, something changed because of something we did. And of course, that's great. But as Piers said, when everybody gets on board with the idea of a portfolio approach, I would like to see more of that because the philanthropic foundations are the risk takers. The governments have way more money and the governments have way more power. So, but the governments have found it trouble to fund the risky stuff. So it's the philanthropic organizations are super crucial because We have to do the risky stuff in order to find out anything. But it'd be nicer if they— if people said, yes, we will do the risky stuff. Half of everything we fund should surprise us, or something along those lines. And then let's connect it to the government. So that's, you know, obviously I'm on a roll with these kinds of ideas today, but, you know, I figured I would continue.
Ruth Levine: Yeah, yeah, yeah. Missionary for— or an evangelist for null results, overcoming Publication bias, overcoming those dreaded funders who are more excited about that kind of linear story. Yeah, really, really great. Dan, you mentioned that in your work broadly, and particularly in Philadelphia, that the most— if I heard you right— the most under-resourced public agencies are the ones who maybe can most benefit from the kind of work we're talking about. So do you have thoughts about either how public procurements or private philanthropic dollars or maybe even corporate dollars can be used?
Dan Hopkins: I certainly have thoughts, and I actually expected you to come to me immediately after you said you wouldn't name any Baltimore institutions, but, but you did. Thank you. I'm not related. The main funder that I often think about is, well, they're legislative bodies, right? Because working with whether it's the City of Philadelphia or the US federal government, legislative bodies are critical. And one thing that I have increasingly felt in my work with the City of Philadelphia and with the US federal government is the importance of of thinking about evidence, thinking about evaluation, not just at the stage of policy implementation, but at the legislative stage, at the stage of policy design. And I've been excited that at the federal level, you know, there's been legislation and a movement in the federal government to do precisely this. So that I think when we are enacting and designing programs from the get-go, that is the time to start to think about how we're going to evaluate them, What data needs to, to be collected? What kinds of collaborations across agencies are needed for that data to be collected? I mean, so often I have found that the city may want to do an intervention, and the federal government is going to know whether it worked, but the data will never be shared back with the city, right? And even within the federal government, it may well be that USDA does an intervention, and the Social Security Administration will know if it worked, But there's no mechanism for getting that information back. So I am— I very much think that we need to think about when we're implementing policies and when we're designing policies, when we're legislating, that evaluation needs to be part of the process. And of course, given political incentives, that's a really hard problem because people want to, you know, take credit for the policy rather than take credit for the process that will evaluate the policy down the road. But I still think it's essential.
Ruth Levine: Yeah, fantastic that you brought that up. The embedding of the— both the earmarked resources for and the legislative imperative for research and evaluation is really, really an important issue and brings to mind the kind of seminal work of Judy Garon and others. So I'm really, really glad that you brought that up. Kerry, anything from your perspective on the funding question? And maybe, I don't even know, have you worked with philanthropic funders in Seattle? There might be a few of them.
Carrie Cihak: Yeah, in Seattle and, you know, other places in the country too. Yeah, I have a number of thoughts. I mean, you know, I can't agree more with Jake and Dan on all the points they made. And what I might add, and kind of picking up on Dan's eaglet analogy, one of the things I feel like we really need is the funding that allows us the time to nurture our fledglings. And often funders really want to fund a well-developed proposal. You have to come to them with a well-developed proposal, and there's not enough funding or kind of infrastructure often in academic institutions that gives us the runway to really build that relationship and really think, you know, nurture that little fledgling so that he can fly when he gets out of the, out of the nest. So I think that's a really important, important piece. We've also, you know, I mentioned Best Starts for Kids at the outset, and one of the things that we did in that was actually a set aside for data and evidence. And I would love to see more set-asides in both, you know, philanthropic funding for program as well as public sector funding that, you know, just creates that pocket of money and expectation that we are going to work on evidence building of those interventions. And I think a really important piece of that and what's been really critical to our success in Best Starts for Kids is that that data and evaluation set aside was not just about funding to contract with outside entities to, you know, come in and do evaluation with us, but it was also really about how do we build the capacity in our community-based organization partners to do the data and evidence work alongside us. It's not realistic for us to put requirements on these really small organizations to give us your data and participate in these, you know, evidence-building projects if we're not really building their capacity to do so. So we put in all kinds of different supports, funding, technical assistance, and, you know, many, many other ways in which we're helping to build that capacity, not only in our government, but in our community. Equity, which is really, I mean, it's, I just, I can't tell you how rich and exciting that has been.
Ruth Levine: Well, go ahead, tell us a little bit about how, that was a real tease there.
Carrie Cihak: Yeah, well, you know, I think, I think there's this common perception that we have to put the equity work over here and we have to put the evidence work over here, and kind of never the twain shall meet. And, you know, there are some good reasons for that tension to exist. There are many communities where, you know, research has done more harm than good and has been very extractive. But at the same time, you know, if we approach it in a really pro-equity, anti-racist way and do it with community. Communities are really eager to use, use their own data. They want to know what's happening in their community. And frankly, it's the communities themselves who have the biggest stake in our success or failure. It's not a politician. It's not a researcher. The impact is in community. And so, you know, really trying to— it's not easy. I'm not saying it's easy. It's really hard, which is why we don't often try to do it, right? But when we do, it just, it creates so much benefit. You know, just a really quick example in Best Starts for Kids, just one of the many things we've been doing is these data deep dives with communities. Where we, you know, we, um, you know, we set these tables— well, now they're Zoom rooms— but we set these tables where we're all looking at the data together, and community is helping us to interpret that and think about what the issues are and what the solutions are. And so we're not just interpreting data, but we're really, you know, setting the agenda for what we, you know, where public investment goes. So it's, it's, it's so impactful, so meaningful.
Ruth Levine: Yeah. Yeah. Really helpful to hear. I'm, I'm eager. Piyush, I'm putting you on notice. I'm going to turn to you in just a couple of seconds to talk about this as a frontier, you know, opportunity and challenge in the work that, that I'm imagining you do. You know, I just want to say that But I've often wondered how our work and certainly our communication would be different if we envisioned developing work that would yield recommendations for action by the communities that are most affected rather than, you know, the policy implications or the recommendations for improved implementation. I think sometimes the recommendations might come out like, vote the bastards out. Rather than the sort of highly technical recommendations that we might generate for the implementers or the policymakers. Really, really helpful transition into this discussion about what community-engaged research looks like, how it plays out differently in different places. Piyush, I'm imagining that as for ID Insight, this is a frontier area for Ideas42. Would love to hear how things— how you're thinking about that kind of community-level engagement.
Piyush Tantia: Yeah, well, I get to open with a hard question, huh? I see. Yeah, it is. It's something we've thought about for a long time, actually. And the way we always did it, of course, we're now questioning and improving upon, was, okay, we're going to start with understanding people's context. We'll go into the field, we'll understand what's actually happening, what their lives are like. And then we'll design something. We won't just take some theory and design. So based on that, and then we'll go further, we'll go and test everything we designed to make sure that it actually works for people and leading to the outcomes that we think they would. And we still do that. That I, I feel is still, is still quite helpful. Uh, we've added a few things to that, which is more co-design where we can do it, you know, without overburdening folks in the community. That's always the challenge. These are often busy people with busy lives, and like, how do you, how do you pull them out for a, you know, day-long design session? You know, it's kind of hard to do that. And who, who comes? All of those are tough questions. Um, some other things we've been doing is expanding our own teams with people who have lived experience and are interested in doing this type of design work. And that's been wonderful. We have a venture studio where we have, you know, 7 entrepreneurs building products for the communities that they come from, and they, the entrepreneurs, come with lived experience. And normally Many folks like this don't get the resources or don't have the privilege to take time off to build a company for 18 months or 2 years. If we were lucky enough to get some funding to support them while they're doing that. And so we're really trying to build up the experience in the team itself as well to see what happens. In the design process, does that change something?
Ruth Levine: Yeah, yeah.
Piyush Tantia: And this connects back to our funding discussion where this was only possible to do because we had somebody give us this type of flexible money at a large scale. It was the Wells Fargo Foundation that is supporting all of that.
Ruth Levine: Yeah, yeah. Well, I think several of you, of you all have focused on the ways in which moving on these frontier areas really does require both flexibility and more funding. So maybe when we're thinking about Piyush's challenge of, well, we don't get, you know, we only get a small fraction of what the physical sciences get, if there were more resources, presumably this would be an area where the work could really be built out. Dan, I'm curious to hear your thoughts about how you've thought about equity. You talked about Philadelphia as being the poorest large city, the largest poor city. I'm not quite sure what the right formulation is.
Dan Hopkins: Yeah, absolutely. I appreciate the opportunity to speak to these issues because, yes, Philadelphia, there's no city that is bigger and poorer than Philadelphia is. We are the poorest of the 10 largest cities. And I think that I just wanted to in part amplify and build on some of Carrie's points, because I think that the question of the relationship between equity and evidence-oriented strategies is really, really important. On the one hand, we have to recognize, and I think we do, that the use of evidence, the use of data, those are skills that certain people have, and that all too often conversations around evidence can wind up with like, men who maybe share my skin complexion, you know, telling— seemingly telling people what to do or saying the evidence clearly dictates this or that. And I think that, um, that evidence then can be wielded as a form of power, as a tool. And I think that that's, that's really, really important to identify. At the same time, in an era, in a city where we're so rightly worried about questions of equity and inequality, I also think evidence has a real role to play 'cause it can help us build a shared vocabulary. The city of Philadelphia has so many ways that we can use the next taxpayer dollar. There are so many valuable opportunities and projects, and evidence then can give us a language to start to think about how we're gonna use scarce resources. And so I think that, and particularly given that the varied backgrounds and languages and cultural backgrounds that people come from, Evidence is at once essential, but we also have to recognize that all too often evidence is used by certain groups in order to forestall, to prevent a broader conversation. And so it's that tightrope that we really try to walk, and I think is so important.
Ruth Levine: Yeah, really, really helpful, helpful point. I also, you know, have observed myself, and I think, Piyusha, you were speaking to this, that the, increased and increasingly successful efforts to have a more diverse and inclusive academic community and community and research and evaluation organizations really accelerates the progress here. And so I think we all have to, you know, sort of put all our strength and resources— again, it does often come back to funding resources in the direction of a diverse and inclusive team doing the research itself. So I'm looking at the questions that have come up from the audience. As you might imagine, it's a pretty smart group, and they're asking nice tough questions. So one of them that I think would be fun to hear from folks on is whether collaborations and these long-term relationships that we've been talking about as key enablers for evidence uptake and the research process itself, has that been, you know, in your experience, has that been sort of easier to come by, easier to develop in some sectors versus others. Certainly health is always out in front. So let's just stipulate health is always out in front on these issues, although not always in the direction we might want. Fyusha, I think you highlighted the lack of social science related to vaccine uptake. Anybody have a—
Carrie Cihak: I can jump in, Ruth. I've definitely found it's easier to develop research partnerships and interest in some fields than in others. And you mentioned health as an example. There are lots of researchers who want to study health, housing, kind of economic questions. It's been a bit more difficult, which is interesting to me, to find researchers and a lot of interest in like some of our public transit research we're trying to do. And I think it's for a couple of reasons. So one is I think it's pretty unusual for a transit agency to really try to own health outcomes. And yet that's what we're trying to do here in King County. We don't, you know, obviously our public transit agency can't control community health outcomes. but we can really own our contribution to those. We know transportation is really fundamental to people's ability to get around, to accessing health services, to getting a job, to getting an education, to social connection. And we're really trying to unpack that. But we've— we have some really fantastic relationships, you know, with institutions I mentioned earlier. But there's not always the kind of the level of interest in that research that I see in other areas. So that's been kind of interesting.
Ruth Levine: Fascinating. You know, there's a researcher, Mahesh Khera, at BU, I think, who is doing work, I believe, in Ethiopia and Malawi on this question of what's the relationship between mobility and some key health outcomes, particularly for women.
Carrie Cihak: So you're going to have to put me in touch.
Ruth Levine: I will. I will.
Carrie Cihak: I will.
Ruth Levine: Yeah. Absolutely. Other experiences to draw on or thoughts about the sectoral focus?
Piyush Tantia: If I may jump in, what Carrie was saying about transportation triggered a thought because I was thinking, well, sector as in different sort of industries or social issues. But I realized where there is a difference is I think in systems change type questions or infrastructure type questions like like transportation, you know, where it's difficult to run an RCT, a randomized controlled trial. It's hard to experiment and get that quantitative data that we've all been trained to love. And we ourselves have had to be very conscious about pushing ourselves to say, you know, actually, if someone is asking us to solve a problem that's a really important problem, just because we can't run an experiment, we shouldn't say no to that project, right? We should do the best we can with the, with the information and knowledge that we have. So maybe that's a possible reason, Carrie, why people don't want to work on transportation, because it takes too long or too hard to run, run experiments.
Carrie Cihak: It's such a great point, Piyush. You know, we need to We need to focus on the biggest problems and solving those and not on the problems where we conveniently have data or research methods that work, right?
Ruth Levine: I would say that the biggest problems and at moments when there are key decisions being made.
Carrie Cihak: Totally.
Ruth Levine: Yeah, that's definitely, definitely. I think there are lots of people who are recognizing that the methods first approach needs to be updated with a decisions-first approach. Dan or Jake, any thoughts about the second row question?
Jake Bowers: Just chime in. I think actually there are many great methods to use to draw impact, to do impact evaluations without randomization. So we, they're not as well known. There's some, you know, they're the, what you think of as the best method might depend on where you got your PhD RCT, but even so they exist. And so there's really active— I mean, I foresee in the next couple of years we will have better and better guidance and standardization perhaps, or at least standard practices about how to do this, you know, anyway, so that we can— it doesn't have to throw up our hands and say if we don't have an RCT, we can't, you know, engage with alternative arguments. And causal inferences. The thing I would mention about where collaborations have really blossomed in my experience has been in the context of networks. It hasn't been sector-specific, but the EGAP organization, you know, puts people in the room together from USAID and from universities. And over the years of doing that, you become friends and you call each other and you say, I need some help. And then you begin to collaborate in, you know, in weak ties or strong ties. The OES does the same kinds of things. I do feel like if you're alone in an agency or in a university, you don't have an opportunity to create the long-term relationships, whereas— or it's harder to develop those. So in any case, I have seen great success with these networks as well.
Dan Hopkins: And I would just add briefly— Ruth, if you want to move on, that's fine now.
Ruth Levine: No, no, no, please.
Dan Hopkins: I would just add very briefly that Universities have well-developed institutions to study certain problems. So there are schools of education, there are schools of public health, and there are often very dense, longstanding relationships between departments of public health or school districts and the associated parts of the university. But there are some problems that are real problems for, say, cities or for the federal government or for states that don't have a corresponding academic department. You know, there aren't to implement extensive studies of litter, for example. And here in Philadelphia, one of the approaches that we have taken is to, um, to be very client-driven. And so we say to the city of Philadelphia, what are areas that you want to learn about that you're not learning about right now? And so, you know, with an economist at Swarthmore, I found myself— and a behavioral scientist at Temple— we found ourselves on the Zero Waste and Litter Task Force here in Philadelphia, focused on litter. And it's not an easy area in which to publish, but it's a critical area that a lot of different communities in Philadelphia say, hey, we need to clean up our streets. And, you know, like, now my family can make fun of me anytime they see any litter that we didn't successfully do our job. But I think that's an example of the way in which there are existing— there's a lot of existing infrastructure in certain areas, and then tremendous opportunities in some others.
Ruth Levine: Oh my gosh, that is, that is so great. Has anybody seen the like Swedish talking garbage can? Okay, we'll send that around. I think that was an experiment in the litter field. Piyush, do you know about that?
Piyush Tantia: Yeah, I do. And the videos are floating around. I'm forgetting now what they're called. I think it was sponsored by VW. There was also a trash can that makes this cool sound of something falling for many, many feet and then going splat every time you put something in there. So just makes it fun.
Ruth Levine: Oh my gosh, so much, so much behavioral insights that could contribute to the litter problem. It's so exciting to hear about. Okay, going back to the— what the audience wants to hear about rather than what I want to hear about. Well, here's an interesting one. So this is about access to data, which of course is fundamental not only to the work that the researchers who are directly involved are doing, but to the public good that can come from broader access to, to data. So the, the point that the audience member wished to make is that large institutions disproportionately have greater access to government data, and that small organizations also need full access to open data to enable fresh ideas for innovation. So taking that as a point of departure, would love to hear any thoughts from some or all of you about the challenge of making data fully available. And we'll just stipulate that funding is one of those problems for data curation and access. Nods around the room. So other than funding, any comments on the issue of data access from any perspective?
Carrie Cihak: Jake, anything?
Piyush Tantia: Yeah.
Carrie Cihak: Go ahead, Jake.
Jake Bowers: I was just going to say that I think that data access is just to reemphasize how important it is if we're going to include the communities in all aspects of our work. There are plenty of people who know how to write computer code and can analyze data, and we should have, you know, we should have participation in all aspects. But if those people don't have credentials to access the data, then it's a problem. Or if the data cannot be shared because of privacy concerns, and that's a problem. I'm hopeful, there are efforts afoot. The Sloan Foundation, I think, is funding some of these efforts to try to figure out ways to share data in ways that are privacy-preserving. Those are, I think, in early days. I see the— I mean, this is why it's very difficult to— this is why we have these models, right? I mean, universities, big universities can make data sharing agreements and can promise that you will analyze health data inside of environments where you cannot copy and paste from into the environment. But if you are a small organization with a laptop and some open source software, which is plenty powerful to do the analysis, We don't yet have kind of like easy, secure data environments that we can allow people to access. Although again, I think people are moving in these directions right now.
Ruth Levine: Yeah, really helpful observation. It becomes yet more complicated in the international sphere where each country has different regulations and they are dynamic. They're moving fast. Other comments about the data question? Carrie, I think I interrupted you before.
Carrie Cihak: No, no. Yeah, I mean, I think some of that goes back to some of my earlier comments about really building the capacity in community organizations to use and work with data. I'd also say that this is particularly an issue with respect to our work to advance racial equity. When you get down to very small communities, that, you know, making that data available publicly can very quickly lead to, you know, people being identified. So we need to, and I, you know, there are many people working on this. I know like the Census Bureau is working on ways to get down to very fine-grained communities and have data about those communities. And we've actually done some quite innovative work in our Department of Public Health that came out of the COVID our COVID response. You know, we have a Marshallese community here in King County who was greatly impacted by the pandemic. And we, you know, we didn't really have great data to work on. So we've got to really solve these problems and solve them quickly and do it in a way where we're protecting people, but also using data to, you know, address issues. I'll share, and maybe we can share with the audience after, we have a great little video we just produced in King County about why data is so important to us and our communities.
Ruth Levine: So yeah, I'm sure people would love to see that. And I also think that there's a lot of progress being made around the ways in which values and subjectivity are embedded in bias in data, the racial equity— I mean, sorry, data equity work that many organizations are now doing is really exciting. Before I wrap up with a lightning round, any other comments on the data question?
Carrie Cihak: No?
Ruth Levine: All right. I'm going to now ask for— quick responses from each of you to the following question. Given that the work of doing research and pushing for the value of null results, developing these relationships, and seeing action results from the work that you do, given all the challenges that undoubtedly you've talked about some and you undoubtedly experience What keeps you going? Dan?
Dan Hopkins: I think it partly, you know, all of us have alluded to this, but the opportunity to make a difference, the opportunity to take these ideas. You know, there's a Shakespeare quote about, you know, I've been thinking too much, I need to do. And I think that there is, there is some of that from my end, that the opportunity to actually see these ideas in practice. And also to, you know, and I mentioned earlier that, you know, professional— it takes a long time to develop the professional expertise to get to the level of, say, a senior civil servant. And to be able to talk, to realize the deep curiosity that public servants have, and to learn from many of the best practices in government, which have been accumulating over decades, but maybe not filtered into the academy. To me, the novelty and the ability to see whether these research ideas work in real-world impactful settings, to me, is tremendously motivating.
Ruth Levine: Great. Apiush, what keeps you going?
Piyush Tantia: Similar. It's the potential. There's so much to do. And, you know, and then I keep thankfully getting proof points, right, because we have projects finish and good things happen. And so that keeps me going, or I hear about something someone else has done, and that's always encouraging.
Ruth Levine: Jake, your thoughts?
Jake Bowers: Yeah, I think that I like to think of myself as a member of a gigantic worldwide team all working to make the world a better place in the face of all the challenges. So when something is going wrong, I think Well, I'll— I'm going to play my role on this amazing worldwide team. And so I love to feel that I'm trying to make a difference when I face the challenges. I also think that it's thrilling to learn both about government, but also to have my scientific theories be challenged by the application itself. So I come away from them thinking, man, we need to go back to the lab. We need some more basic science here. What's the science of federal form filling out?
Carrie Cihak: Out.
Jake Bowers: You know, what, you know, it's that we— so there's so many new questions coming up that it's also exhilarating in kind of a scientific perspective in addition to making a difference.
Ruth Levine: This is a man who just loves those null results. Carrie, I've heard a lot about what keeps you going, but would love to hear you sum it up.
Carrie Cihak: Yeah, well, I'm definitely on Team Jake. You know, as a public servant, it's really I talk about how my richest, best education is not, you know, in the academy, but in through the work that we've done with community here in King County. And it really is those communities that keep me motivated, and particularly the communities who we've most marginalized. They deserve our deep commitment to build evidence with them. And my experience is not only that they're eager to do so, But they, you know, they want to be there with us. And they face, you know, as I said before, they face the highest consequences when we don't have that commitment. And so we have to follow through on it.
Ruth Levine: Really, really wonderful remarks. Thanks to all members of today's panel for this really enriching and inspiring conversation. I'm sure both experienced researchers and the next generation has learned a lot from this conversation. I want to again thank the event's co-sponsors, EGAP, Ideas42, IDinsight, and Sage Publishing. And a heads up for the audience, the information about Social Science for a World in Crisis, including how to view previous episodes in the series and find out about about upcoming episodes is coming on your screen in just a few seconds. Thanks again to the panel for a great conversation and to everyone for joining us today.
Narrator: That was Ruth Levine, Jake Bowers, Kerry Cheok, Daniel Hopkins, and Piyush Tantia. As always, you can follow us online or in your podcast app of choice. And if you're interested in learning more about the Center's people, projects and rich history, you can visit us at our website at casbs.stanford.edu. Until next time, from everyone at CASBS and the Human Centered team, thanks for listening.