For the full interview please listen to the most recent CM Group- Free Lunch Podcast, Episode 78 “ ‘Superforecaster’ Dr. Warren Hatch”
Colin and Greg: Tell us your story! How did you end up here where you are today?
Warren: Well, it was kind of on a lark. The author of this book, Super Forecasting, along with Dan Gardner, is Phil Tetlock. He's a very distinguished psychologist, and he had written an earlier book called Expert Political Judgment. Political Science was my old academic background and I really liked that book. The point that he made was a lot of experts will make forecasts, but when you go back and match their forecasts against reality, they're not so good. He documented that with a lot of evidence and empirical data. The one thing that he also found was that on shorter horizon, some people actually can make good forecasts. So that's what he started planning next.
A decade ago now, the U.S. government had just come off of some pretty important forecasting failures. With 9/11 they missed, and weapons of mass destruction that they thought existed did not. They were getting type I and type II errors that were very consequential and did some serious searching for ways to improve their analyst capabilities to make forecasts on shorter term horizons. Improve on the wisdom of the crowd was what they set out to do. The government sponsored a competition among five university based teams, one of which was Phil Tetlock and Brad Miller's team called “Good Judgment”. It was right around then that I thought, “Well, I wonder what Phil's up to. I liked his book.”. I went to his web page and down at the bottom he had an announcement looking for volunteers to join this new research project. I signed up having no idea how big the project was or how deeply involved I would end up becoming, and I went from there.
Colin and Greg: Wow. So on your background, you were in the investment business with Morgan Stanley and you know what it's like dealing with individual investors, and we all want to know what's going to happen. The basis of investing seems to be: “Where are things going?”; “How are we going to predict what's going?”; “Is there going to be inflation or interest rates going up?”. Things like that. How important is it for investors to understand forecasting and how can “super forecasting” help make better investment decisions in your opinion?
Warren: I wish I knew then what I know now when it comes to forecasting because there are so many things that I learned to do that I had to unlearn to do later on. One of the big things is that forecasting and finance often is done very generically. So is there inflation? I think there is, and you might select a few anecdotes and be very focused on what's going on at the moment. But what we know from the data is that you want to go and find comparison classes. What kinds of similar environments might we find that will tell us something useful from the past or other countries will tell us something useful about what we're trying to understand today and that very basic step is something that can be very helpful for any investor, individual, professional, institutional. You start with a base rate because all too often people will say this is special. This is different. My security is special benefits that no one else has. Don't bother me with the macros, don't bother me with the industry group, but really, that's a conclusion to be made, not an assumption to start with. So something that fundamental, I think, is very powerful for all investors.
Colin and Greg: Just for our listeners. You're talking about general forecasting. As Greg said, we're in this environment where people are looking for certainty in what is forever an uncertain world. They talk about, “What's the market going to do next year?”, and we often talk about how things like the past can't repeat itself, but it certainly can rhyme. What is the difference between forecasting and can you describe exactly what “super forecasting” is to the layman listener?
Warren: For me, forecasting is simply thinking probabilistically about the future. Very few decisions we make are going to be an absolute yes or an absolute no. They're going to be within the range somewhere in between. It's a world of “maybe” and what we want to do when we think probabilistically is be as precise as we can. That's why we use numbers instead of words. So will there be inflation? Let's turn that into something more precise. Will inflation a year from now be more than three percent? So we've got a date, we've got a timeline, now we can express it with a probability. Let's say it's 87% probability that that will be the case, that becomes meaningful. That's something precise. We can all understand what that means. We can keep the track record of whether I'm right or you're right or Greg is right and then get more confidence in the forecast we're all making. Forecasting is just thinking probabilistically about the future. Really, every decision in that sense is a forecast, and what “super forecasting” is about is a data driven empirical process that we know leads to better forecasts, both for individuals and for teams. There are steps that anyone can take some simple, some more complex, but all of them can have incremental improvements to the accuracy of our forecasts based on what you might otherwise get and by accuracy. I should emphasize too, is you're getting the best possible forecast right now. If you can get the best possible forecast right now before the competition, you have time and having that time advantage is why it's really worthwhile. If you're an investor and you can beat the crowd to the best possible forecast, that's where profit is.
For more questions and answers with Dr, Warren Hatch please refer to our Podcast.
Stay safe, stay happy, stay well!