Robert de Neufville, Writer and Superforecaster

92-126. Robert de Neufville

2-126. Robert de Neufville
SPEAKERS
Will Bachman, Robert de Neufville, Speaker 1

Will Bachman  01:02
Hello and welcome to the 92 report, conversations with members of the Harvard and Radcliffe class of 1992 I’m your host, will Bachman, and I’m delighted to be here today with Robert de Neufville. Robert, welcome to the show.

Robert de Neufville  01:16
Thanks for having me on, will, and thanks for doing this incredible and I want to say kind of insane project that you’re doing. It’s, it’s super valuable, yes, exotic is, is very insane. I’ll take that. So tell me about your journey since graduating from Harvard, yeah. Like I knew you were going to ask this question as the whole point of it, and I was thinking about it last night. It’s a huge, daunting question. Like, I don’t to try to give an accounting of, like, the last 30 years. Like, I don’t even know if I have a story to tell myself. That’s very good. I just kind of muddled through and, you know, looking back, and I’m like, What? What was all that like? I’m not really sure. I’m hoping that your AI, transcript, summary, or whatever, will have an explanation for what I’ve been doing, I think the short answer is I ended up spending a lot of time in therapy like that wasn’t my plan. I didn’t want to spend my life that way, but I felt like it was something I really needed to do, and that was to the extent that I put a lot of my life on hold for a while. I didn’t do a lot of other things. I wasn’t really able to come back from a therapy session and then just go and do normal work. So the result is that I feel like in some ways, I’m only just, or only relatively recently, getting started on some parts of my adult life, even though, obviously, you know you and I were not really young anymore. So like, in terms of, like, having a career that I feel like I connect with and family life, those are just things that got put on hold for me, and I feel a little bit behind. In some ways, not to like, be judgmental, not to like, be down on myself, but like it honestly, I experienced it as a little bit of a loss like that wasn’t my plan. I did. I had other things I wanted to do, but that’s just what I had to do, I think.

Will Bachman  03:19
All right, tell me and tell me about like, where you are now,

Robert de Neufville  03:26
yeah, well, so now I’m I live in Hawaii and moved out here, dropped out of grad school after spending more than a decade in grad school and ultimately not finishing my PhD, and did that right around the time of the financial crisis, and it was a terrible time to do that, to decide that you’re starting over and try to look for work, and one of the things I realized is that you’re actually a lot less employable after a decade in grad school with No credential to show for it, but I got a, you know, a master’s along the way. But like, you know, having spent all that time in academia made me less employable in some ways, certainly, than I was after we graduated from Harvard, when, you know, lots of jobs were available in some sense. So it was really, it was difficult, and I didn’t really know what I was going to do next, and the job options for someone with my skills. I’d done a lot of teaching. I was at grad school at Berkeley, done a lot of teaching there. You know, I had options to teach political science classes, but adjunct teaching is just a terrible, terrible deal, and there were a few good jobs. I ended up teaching a Berkeley program that paid pretty well for a while, or I taught at San Francisco State for a while, and that was a, you know, something sort of close to a living wage. But I would get, you know, offers to teach. At a community college, a semester of a class for $1,000 you know. And imagine how many of those classes you’d have to teach in order to support yourself in in the San Francisco Bay area. So it was a honestly, it was a humbling experience. Like, I’m like, I’m really good at some of this stuff. I’m very knowledgeable and expert in some areas, but I suddenly found myself really not known what to do, and did some freelance editing various different types of projects. And I say I ended up moving out to Hawaii, really, because I had a friend here. Well, I liked it. I always liked it was a kid who came here as a kid and was really attracted to it. But I have a friend here who is at u h University of Hawaii and had a an in law unit, an ohana unit in the back of his house that he wanted to rent out because no one can afford a house, so they all rent out parts of their houses. And he was opening it up. And I thought, well, we’ll just go move in there. I can do random freelance work as well from Hawaii as I can from the Bay Area. And I needed to change. So I’m here in Hawaii now, and, you know, we can go back maybe at some point to, like, the first part of my, you know, journey after college. But what I’m doing right now is I’m trying to write, I have a sub stack newsletter called telling the future. I am a forecaster now, and I can explain sort of what that means, or how I got into that, but I write about forecasting. I write a lot about politics. As I say, I did graduate work in political science, and I’ve always been interested in political science and political theory. So I have this newsletter where I write about those things. And I love writing. I love that. So I’m trying to figure out how to make that a sustainable career, which so far it’s, you know, fairly far away. I have something like 1500 subscribers, which seems like a lot to me in one sense, but it’s nowhere near enough to make it really a sustainable job. So that’s what I’m trying to do. And of course, you know, you talked about, we’re all about self promotion. Please go to go check it out, subscribe, boost my numbers. For me, I think it’s pretty good that said, like I write about politics, and I’m not super happy writing about politics right now. Part of me would like to not think about any of it, but it sort of seems like that’s what I need to do. But it’s kind of dark time to write about politics, and I kind of wish I wasn’t doing that, you know, although I think people also appreciate it. So that’s where I am. That’s where I’m right now, I guess is the answer. Do

Will Bachman  07:55
you want to tell me a little bit more about just that first period after college and therapy? I mean, I don’t want to pry on it, but is there, you know, is that a topic that you’re comfortable discussing? Yeah. I mean, I Yeah,

Robert de Neufville  08:11
it’s there’s kind of a stigma about it, and I think it’s important to talk about and to normalize in some ways. Yeah, I agree. But I also, if we’re being honest, I’m a little embarrassed about it. I’m not proud of it in certain way, like, as I say, it’s not really the life I wanted to have. I envisioned for myself, doing other things and and also, you know, I don’t want to, like, I don’t know, maybe talk about parts of my family life or something that, because they that, but like, fundamentally, I just kind of reached a breaking point. You know, I wandered around for a while. I didn’t know what I want to do after college, I graduated, put all my stuff in a U haul, drove down to New York with Christine van Kipnis, and, you know, arrived in New York never having driven a U haul, and finding it very scary to drive through New York and try to, like, stop a U haul, 12 foot long truck. But I know I was going to do, I ended up, I feel like doing kind of random things. I worked for a while in designing an LSAT course with a company that taught Test Prep. Did test prep stuff on Long Island. Then I mentioned to you I did a couple of years as a management consultant with Booz Allen and Hamilton’s media and entertainment practice. Really liked making consultant money and feeling important in a certain way, but I just couldn’t get very excited about sort of fractionally improving Viacom’s net margin that just wasn’t that didn’t have any meaning for me. So I stopped doing that, and I really. I wanted to find something that mattered to me more intellectually. I really like thinking and writing about things that’s sort of been a through line through all of this. So I ended up applying to grad schools in political science. I had always gravitated towards political theory. I think I’m kind of, I want to say a political theorist by vocation. That’s actually a reference to a fairly famous political theory article, but in the sense that I think about things in the way that political theorists do, and I mean, it’s a little bit different. It’s sort of, you know, when they teach political theory, political theory in political science departments. It’s kind of hybrid between philosophy and intellectual history. And that was always sort of where, what I was interested in thinking about, sort of how we live together in communities, and moral issues involved with that and doing it in the context of history. So I ended up going to grad school at Berkeley, unfortunately, sort of being a political theorist by vocation is not really the same as wanting, or, you know, being a good candidate for being a political theory professor. I ended up finding that academic culture really wasn’t for me either, and I wasn’t very good at a lot of the academic parts of it, and found a lot of the environment fairly toxic. It’s sort of weird to me. One of the things that struck me is that, you know, you go to a good school like Berkeley, the professors there are incredibly accomplished, smart, successful people, and some of them are lovely, but many of them are also just super insecure and mean, in some way, you know, they need to be better than you, and it’s pretty toxic environment. And in some ways, I had both my parents professors, and I had followed them into academia, even though consciously not really wanting to be like them. And I think for me, some of the toxic professors really reminded of me, me of my toxic parents, and I was like, I can’t, I can’t really do this. So after a few years in grad school, I kind of reached a breaking point, had a bad relationship, and that really wasn’t that was more maybe a reflection of what was going on with me than the cause of my breaking point, but I felt like I couldn’t really go on the way I had been going on. I wasn’t really getting much work done schoolwork. I was always been a night owl, but I would literally stay up all night, every night Get to bed after the sun came up at 7:30am and, like, even if you have a late Chrono type, which I do like, it’s it’s a terrible way to live. You can’t interact with normal society that way, and you don’t get enough sunlight. And it all felt kind of crazy, like I think I was staying up all night, because at night I didn’t, I was, like, avoiding the regular world. And during, you know, I could just sort of sleep through the day part of it, you know. And I would get to my office at like, 4pm and, you know, an office on campus and try to write my dissertation, and go around in circles for hours, and then come back and midnight and have a late dinner, and it was didn’t feel like a good life, and it wasn’t healthy. It really wasn’t healthy. I would have fairly routinely severe panic attacks. I think I’ve had those as long as I could remember, but I didn’t really know what they were. I just thought that everyone had something like that, like, I didn’t realize it wasn’t like you see in the movies where suddenly you’re, like, breathing hard and had to sit down. I would just kind of, you know, start to sweat and have my blood pressure spike, but I would kind of go on with my life, just a normal part of my life. And, you know, grad school can be pretty depressing, like it’s actually shockingly common how unhappy a lot of PhD students are. I think it’s a very depressing it can be a very toxic environment. So that didn’t really help, but that really wasn’t the issue. I just I had a pretty kind of difficult childhood. My parents were not around, not that interested in taking care of a child. Uh, fairly narcissistic, and it kind of got worse in some ways as I got older my my mother really would become very abusive, you know, the extent that I tried to live my own life and not have my life focused around her to say awful things to me. And I finally, I don’t know, checked myself into a therapy, basically. And once I started, I was like, I really need this. You know, I would go to a therapy session and just come out of it absolutely devastated, like, I mean, it felt like I was making progress. And certainly in hindsight, I feel like a much better relationship with the world and myself than I did. But like I would, I would go to after my therapy. Sometimes I would go to the cheese board in Berkeley, a pizza place. They have really, really good pizza. And I would sit there and, like, have couple of slices of pizza, just kind of like quietly cry. And I think they all sort of, they saw me every week do this, you know, I think they felt sorry for me. Oh, very nice. And they’re super nice there. But like, I couldn’t, I would come home and I wouldn’t get any work done after that. And, you know, I was flexible, because I was just supposed to be writing my dissertation, and my professors, they weren’t too concerned if I just disappeared. It wasn’t their problem. Maybe another thing I didn’t like about grad school, so I just kind of sort of quietly, you know, like quiet quitting. But I sort of just quietly dropped out, like I was doing stuff, but not very much. And, yeah, I tried some medications, most of that didn’t really help me at all, mostly with the talking therapy that helped, I ended up being diagnosed with chronic PTSD from childhood trauma, basically. So that’s what I did, and that was like, I don’t know, a decade of my life, wow. And, you know, our peers were having kids and had careers, and so they were doing well. And I don’t think anyone’s life necessarily takes a straight line. I’m sure it’s all more complicated than that, but like, to me, it really seemed like I had fallen a life stage behind where our classmates were at, and, yeah, I felt I didn’t love that. Like, I feel some sense of loss about it. I feel a little bit like, you know, I had this childhood that was unhappy, and then to some extent, part of my adulthood was lost too. Just trying to figure it out. So, you know, I said I tabled a lot of grad school, and I did some teaching. I did a lot of teaching. That’s what I did. And I like teaching a lot, except for the part where you have to grade students. Nobody likes that. They hate it. I hate it. And, you know, Cal, I mean, I think this is more and more true, but some people were really had a very unhealthy relationship with the grades. And I, you know, at some point there were couple people I felt I had to refer to, like, suicide watch, basically, because they’re like, if I don’t get an A, I’m not gonna get into Harvard Law. And, you know, you could see them shaking, and I’m like, Well, I mean, your essay wasn’t that, you know, was a minus, I don’t know, like, it was that part of it, like, but teaching is great. It’s great in the sense that, for me, anyway, I feel like I never really understand a topic until I try to teach it. Sure, what you really learn about is when you think about trying to explain it to someone else. And I don’t think you can really, you really understand something unless you can explain it to a smart, non expert, you know, and then, you know, some people really get a lot out of it. Not every student appreciates your course, but I love that part of it, but that’s basically the one thing I was doing. I wasn’t doing much else, and realized I didn’t want to do academic work, that I wasn’t someone who was going to write a ton of papers and go through the cycle. And honestly, political theory is the worst thing probably, to come out with a degree in there’s this backlog of people who want to be political theory professors. And they, you know, spend their time teaching adjunct and spending all their money on, like, fees to go to conferences, to try to get get a job. And it’s basically, from what I can see, pretty horrible. I.

20:00
My

Robert de Neufville  20:01
advisor encouraged me to just sort of power through and get my dissertation done. And I feel like I should have, I would have loved I put a lot of time and effort into grad school, and I’d love to have a credential to show for it. I mean, you know, it’s this big gap in my resume on some level, you know. And I feel like I know all this stuff my you know, Masters I got in 1996 doesn’t really reflect that, but I just didn’t, I just didn’t have it in me to power through it. And I sort of lost interest in it. I was like, I figured this out to my own satisfaction. And I don’t really want to go through all the hoops of writing this. I mean, a dissertation is not a good piece of work, right? Like, they make you do it in such a way that it’s you’ve, like, pro forma, you hit, you hit a lot of things, but, like, it’s not something that you would ever want to read or revisit for the most part. And I just didn’t, I didn’t have it in me. Yeah, you said that.

Will Bachman  21:01
You kind of felt a sense of loss because you had this sort of vision of what you wanted your career to be, what what was that? What was that vision? What were you imagining how it’s going to play out?

Robert de Neufville  21:15
Yeah, that’s a great question. Like when you when you ask it that way, I’m not sure I had a very clear vision, right? And I almost think it sounds pretty, pretty, you know, vapid when I think about what I thought. But I imagined that I would sort of follow some kind of normal career path, what I want to say as a normal career path, what I thought of as a normal career path, and be successful and accomplish some tangible things that, you know, I would maybe come into this interview now with a like, here’s the arc of my career. I’ve always been this. I did this step and that step and got to this point, and I don’t feel like I did that. I feel like I took this detour and they just kind of moved on from it, and it’s more complicated than that, right? Like a grad school was very intellectually rewarding, and my training and political theory really shaped who I am in some level. I don’t feel like any of that was wasted, but like, if I had, if I could choose again, maybe I would have taken a different strategy and built towards something that I wanted to do now in a way that I didn’t do. So,

Will Bachman  22:31
yeah, what’s the ideal next decade look for you like now, you know, kind of with the revised state? Yeah,

Robert de Neufville  22:41
well, I don’t know. I mean, I really like to write, and I like to think about things, you know, I talking about teaching and how you really understand something when you think about explaining it to other people. And that, to me, is a lot of what writing is, I like to get to the heart of some complicated issue and be able to explain it. And it’s satisfying for me when somebody else appreciates that. But a lot of it’s just for me, you know, I like that discipline is exciting. I like working through things, and I like language, so I like expressing it, and I want to figure out how to make that a viable career option. I think that’s really hard. I think I don’t know if the stats are right, but I saw something, you know, the number of people who actually make a living off their writing, as opposed to other things, like maybe teaching writing classes or something is really small, like, the equivalent of being, like, a, you know, number of people who are billionaires or something, and, you know, I don’t know if that’s something I can really hope to get to, but like, that’s what I would like. I would like to be able to just do my writing and thinking and reading and have that somehow be, be a career that that pays me. And as I said, I’m not, I’m not there yet.

Will Bachman  24:08
You know, I also do tell us about one or two of the either articles you’ve written for, you know, you I see that you’ve written for mainstream like the economist, the national interest, California magazine, The Washington Monthly, either a piece that you wrote for one of them that you personally found really satisfying, or maybe something from the telling the future newsletter, some issue that you’ve tackled that, uh, illustrates that sort of sense of flow that you get. Yeah, well, you know, a lot of those things that I say I wrote for those places, they’re all true, but it doesn’t, they don’t necessarily see my bet, like my best work, and like the economist, I didn’t write anything that got published in The Economist magazine. I was, I did a, you know, online blog for them for a little while. Alright, didn’t really, you know, I.

Robert de Neufville  25:00
Found that a very difficult experience, actually, I blogged for a while for a website called Big Think about politics, was writing, somehow, writing, like, three pieces a week. I don’t know how I did that, because I don’t write. I’m very slow. Usually I’m very fussy and and it’s a real problem for me. I had to, like, crank out more stuff, but, you know, I’ve been doing a lot of I’ve been doing a lot of that for years, and I would occasionally get invited to write a book review or something. Washington Monthly was interesting. I had a friend who had worked with them, and he basically said, they won’t pay you. They they’re going to say they’re going to pay you, but you have to beg them for your money. You have to repeatedly call them over and over again to ever get the $300 that they’re supposed to pay you for because they don’t have money. So their strategy is basically to make the writers beg for it, so that they only pay writers who really, really need it. So if you’re, you know, a famous writer, and you wrote a book review for them, you’re never getting your money because you’re just not going to bother to pursue that $300 and you know, I found that all the whole process is pretty frustrating, where you have to, like, you’re getting $300 for a piece of work that took you quite a long time to write. And I mean, even just reading the book takes, takes a little while for a book review, and, you know, you still have to beg just to get paid for that work you did. So a lot of that was like that. Freelance writing is not very rewarding. Now I’m writing a sub stack, and I’m working more in a subscription model. And right now, everything is I write is free. Nothing’s behind a paywall, which maybe I’m going to have to change. But, you know, I get people who want to support my writing to pay for it, and that’s actually really rewarding when somebody’s like, here, I want to give you $50 to keep writing so and by the way, you know, with our traditional media dying, and if you’re, you know, canceling your subscription to the Boston post, I highly recommend you support some writers you like, if not me, then someone else or I, you know, I donate to ProPublica, which I think is fantastic, but, you know, pay for the right and you love, I would say to do that because it’s, it’s hard, and it’s, it’s, it’s not very rewarding. It’s hard to make work. But you asked about a specific piece. I don’t know. I’ve been writing. I wrote one recently about what’s happening in in US politics. It’s actually the hook. Was about a the play rhinoceros. It’s an Ionesco play in which people turn into rhinoceroses, one after another and become sort of rampaging roots crashing through everything. And it’s a parable about fascism, and that’s sort of my, you know, unfortunately, where I think we’re at. So I’m proud of that piece. I got a really good response. I write a lot about forecasting. Another thing we could talk about is that I have, along the way become a sort of accomplished, skilled, what you would call a judgmental forecaster, which is to say that I use my judgment to estimate the probability that things are going to happen, and I’m good at it, like statistically, I’m good at it. So I write a lot about forecasting, and one way I make money, besides from my writing, is producing forecasts for a number of organizations that now do that. It’s sort of a relatively new

Will Bachman  28:52
kind of forecast in your field. So that’s about part of my writing as well. I’d love to hear about your approach there. Maybe you can make it specific and talk through some specific forecasts you’ve made and sort of what your process was,

Robert de Neufville  29:06
yeah. Well, in general, I can’t think of a you know really well, maybe I could. But in general, the the best way to best guide to what’s going to happen in the future is the past. And people who are good at this kind of judgmental forecasting, there’s a technique they use. And if you ask me, like, what’s going to happen in an upcoming election, the first thing I would do is I would look back at similar elections and ask what has happened in those and sort of you establish basically what you would call a base rate, how often does whatever you’re asking about happen in that kind of situation? So that really guides you in terms of estimating the probability that whatever you’re asking about is going to happen. But. Yeah, it’s more complicated than just doing that. It’s often difficult to figure out. Well, make it, make it specific. For me, I’m a bottoms up guy, so just your example of one forecast you’ve made, and how you researched it, and how you came up with your forecast, 67.3 or something. Yeah. Well, you know, it’s funny. Good forecasters are able to be somewhat precise, but I would probably not use a point three that’s too, too precise for for, you know, really, my, my, I don’t have that kind of fine grain resolution on probabilities. Well, you know, I didn’t do great in forecasting the recent presidential election, I did a really good job of forecasting me the previous midterms.

Will Bachman  30:48
What was your forecast? I mean, for the Trump Biden? Oh, I, I,

Robert de Neufville  30:57
I thought Harris was more likely to win than Trump. Somewhat more likely. But like, I in retrospect, I think that was wrong. I think that I was missing. It’s hard. You can’t say, if I say there’s a 70% Harris wins and then she doesn’t win. You can’t say that was wrong. Maybe the 30% thing happened, right? Like, but you know, in order to know whether I’m doing a good job, you have to have a bunch of forecasts and see if I’m but in that case, in hindsight, I think I missed some signal. I was surprised. I was very surprised that that Trump’s victory was so across the board, like in terms of of increasing votes, sort of everywhere and in groups, I didn’t think that he would increase with so I I really didn’t see that coming. And I think it’s harder to forecast things that you’re close to in a certain way. I have feelings about, about the election, what I wanted to happen and, and, you know,

Will Bachman  31:57
give us an example of, you know, some more, like, random kind of forecast you made, industry or sports or whatever, and kind of how you went about it.

Robert de Neufville  32:11
Well, it’s what I don’t remember as many of the random ones. But I would say, before I got involved in this, there was a thing called the good judgment project where, you know, that’s where they I performed well and realized I was very good at this.

32:27
This is, like the Tetlock thing. The Philip is the Phil Tetlock thing, right? So I’ll talk

Robert de Neufville  32:34
about that in a second. But just to answer your question, before I did that, when I was supposed to be writing my dissertation, I got involved in an early predecessor of like FanDuel, like a sports, fantasy sports kind of site. And the reason I thought I would be good at the forecasting in general was that I basically initially won all their competitions. It was a small group. The reason I did it was because I was one of the only people who was really the idea was to predict, you know, how many receptions a wide receiver would get, or something like that. And I think I was one of the only people who was doing it in a systematic way. So I would look at past history and make projections and build little models to try to figure that out. And I think most people were just there saying to themselves, I have a good feeling about this. You know, it looks good this week. And so a lot of it is just, it’s just that looking at past performance and trying to extrapolate that into the future. And there’s a lot of judgment that goes into that, and you have to ask yourself, Is this player injured? And what are the other factors and and that kind of thing. But fundamentally, that’s what it is. It’s just using past, similar past circumstances to try to project into the future. I mean, there are some things that you have to project more analytically. There’s not a good you ask me, Are we going to, you know, figure out, discover AGI in the next few years? It’s hard to say, because we’ve never done that before, so there’s no precedent. And you can maybe look at other technologies and say, Well, what’s it like to invent those other technologies? How often has that happened? But for that, you have to use a lot more analytic reasoning. But for things like, how many receptions is a wide receiver going to get? You just look at how they’ve done it in the past, and like, you know how good the defense they’re playing is, or something like that. So that, I don’t know if that answers your bottom up question, but that’s the fundamental method.

Will Bachman  34:34
Okay,

Robert de Neufville  34:37
as far as the the Phil Tetlock thing, yeah, there’s so there was a government agency called IARPA, the intelligence Advanced Research Project Agency, which I guess is the cousin of DARPA, which is sort of famously helped invent the internet. They set out to figure out if anyone could forecast geopolitical questions. And I. The research had shown basically that people were terrible at it, the general, even like analysts and pundits, especially pundits, or have an incentive to produce hot takes, were just basically terrible at it, like you would be as well off just flipping a coin as listening to most forecasts. But what they did find when they did this tournament was that a certain percentage of people were able to consistently outperform even like intelligence analysts who have access to classified information, and they did it through using the kinds of techniques just the methodical extrapolations of the past that I just talked about. So I participated in that tournament. I heard about it on NPR. I thought I’d be good at it, and was good at it, qualified as what they call a super forecaster in in my first year, and ever since then, I’ve sort of kept doing that. That’s the tournament is over, but there are a lot of other projects I work with metaculus. Sometimes I work with that was called the good judgment project. There’s now a spin off professional good judgment Inc, I work with them. I work with a group called the Swift center, and we try to produce probabilistic forecasts for people who want to know what’s going to happen. It’s a little complicated, because, you know, when they found out that people like me could, you know, accurately say that something was 70% likely that everyone’s like, Oh, that’s great. We can do all the stuff with it. But it turns out that decision makers don’t know what to do with that. Like, that’s not all that useful to just put a probability on something. So a lot of what we’re doing now is trying to figure out how to make forecasts useful. Like, let’s say you wanted to, you know, make COVID policy a global pandemic has hit. You don’t want to know like, what’s the percentage chance that a million people are going to die? You want to know something like, how many lives will I save if we put in a mask mandate? So you have to ask kind of different questions you might want to ask, you know, how many people will die if there’s no mask mandate? How many people will die if there is a mask mandate, you might want to look at it from a whole bunch of different angles related to policy. And, you know, understand the reasons why people get their forecasts as opposed to just the number that I’m good at producing. So that’s sort of what I think the forecasting community is trying to figure out how to make it useful, and that’s one of the things I write about a fair amount.

Will Bachman  37:38
So my understanding is that, you know where that work of Philip Tetlock is done, and sort of documenting the processes of super forecasters that the knowledge is sort of how to do, it is becoming widely disseminated. And something like, Oh, if you’re trying to forecast how many receptions, maybe you should look at how many receptions the guy got, like, the last 10 games, you know, and do some research people are aware of that now. So what is it? Do you think that, like, what is the art of it that differentiates you to still be better at it than other people that also can look up the number of receptions that the football player got, you know, like, what? Yes, what? What do you feel is your secret sauce that that differentiates you?

Robert de Neufville  38:31
Well, I think a lot of practice, there’s also, I think, some maybe natural ability for it. It’s the thing about it. I think the reason why, you know, it’s called the good judgment project. There’s a lot of what you call judgment that’s hard to quantify in some way, a lot of skill that’s difficult to explain, which is why you can’t, you can’t make a brute force algorithm which will tell you how to forecast elections. I mean, you can have models, and they perform some of that function, but like it’s their guesses fundamental, their guesses about what matters and what doesn’t. So it’s a little bit like, I could explain to you, well, I probably couldn’t explain to you, but like, someone could explain to you how to shoot a free throw accurately, like, what form you need to make the ball go the same direction every time, right? But like, if they explain that to you, that to you. That doesn’t mean you can just do it right. It’s hard to execute. So it’s a little bit like that. The the process of executing you and you know the technique, and you can certainly work on your form at forecasting and get better at it. Anyone can get better at it, but the process is hard to execute, and some people have more, I want to say, natural ability. Some people have practiced more at it. You know, in basketball, some people are good at basketball because they’re taller, more athletic.

Will Bachman  39:50
People who are good at forecasting. What’s that? I’m curious if you have some kind of log book where you keep a log of all your forecasts and then after it happens or doesn’t happen.

Robert de Neufville  40:00
You write down. Okay, here’s my lessons learned, and what did I miss? And, you know, doing some kind of critical review process, yeah, yeah, yeah. So I think, I think it is the intentionally practicing in that way helps. I think one reason I was good is I’d kind of done things like that throughout my whole life, just, I don’t know, because I’m neurotic or something I, you know, it’s something stressful for me. I want to figure out how likely it is to happen. I’ve always kind of done that, but, yeah, you know, during the good judgment project, they did all the record keeping for you. But now I do it. There’s a website called Fatebook, which I use to log my forecast, a link to it, to my my my sub stack newsletter. And, yeah, I do ask myself, you know, why did I get that wrong? Like, what should, in hindsight, what probability should I have assigned to Trump winning, given what we knew and and what was possible to know? And I think that going through that exercise really does help improve you next time you get a situation, you’ve learned some lessons from your previous forecast, right or wrong, that you can then apply to future forecasts. I think that’s that’s right.

Will Bachman  41:16
Tell me about any courses or professors that you had at Harvard that continue to resonate with

Robert de Neufville  41:22
you. Yeah, you know, I want to say, I think the thing at Harvard that had the biggest impact me, on me was our classmates, like I really, I came out of it. It’s just really exciting to be among such a brilliant and interesting group. I found it really, really enriching. I liked my classes, but for the most part, they weren’t as big a part of my my intellectual life. In some way, I did really appreciate Stanley Hoffmann’s class. I think I don’t remember it was ethics and international relations or something like that. I remember the title I liked that. I mean, obviously I said I’m interested in political theory. That was really right up my alley international relations as well. But he, he saw things and taught through a very humanist lens, looked at the impact and in a moral way, and that was very important to me. He also, you know, there was a list of movies you could watch along with the course. And, you know, they both literature as well as, like, academic articles. And I love that. That’s how I think that really informed my teaching. To some extent. My writing always involves, you know, not just academic theories, but art and jokes. And I think kind of in a sort of some kind of bricolage or something, but some kind of, like putting those things together in that way. I also want to mention, I just listened to your interview with with Jennifer Gibbs, and she mentioned Adrienne Kennedy’s playwriting class. And I didn’t get to know Adrienne Kennedy the way some of my classmates said, the way that she did. But like that was it was a great class that was really and for me, as much as as the relationship with with Adrienne Kennedy, it was really exciting to be in that class with people like, like Jenny, like Eisa Davis, just brilliant people working on these things. And it was just really exciting environment. And I don’t know I’m not that great an actor. I did a lot of acting. I’m not that great an actor, but I love to do cold table reads. So reading people’s plays fresh off the presses, that way it was, that was a was a pretty great experience. Yeah. So I guess those are, those are my answers.

Will Bachman  43:39
Okay, cool. So you mentioned your sub stack. Let’s lay out it again. Make sure people don’t miss it. We’ll include the links in the show notes. Where can people find you online? They can find me at tellingthefuture.substack.com