Models vs. Markets: Forecasting the 2020 U.S. Election

Abstract

We present a case study of forecasts for the 2020 U.S. presidential and congressional elections. Specifically, we investigate the accuracy of two competing forecast methods: model-based, statistical projections (from FiveThirtyEight.com) and prediction market prices (from PredictIt). We propose a market-based scoring (MBS) method for evaluating the performance of probabilistic forecasts. Our analysis finds that PredictIt and Five Thirty Eight perform comparably based on traditional metrics such as calibration and accuracy. For market-based scoring, however, we find that, if we ignore PredictIt’s fees and commissions, then Five Thirty Eight forecasts beat the markets overall, but if we factor in fees and commissions, the markets beat Five Thirty Eight. We discuss the implications of this analysis for forecasting future election cycles and betting market design and operations.
Comment: The first version of this article was posted on Researchers. One on October 26, 2020, 1 week before the 2020 U.S. election. That first version serves as a pre-registration of our proposed method before the election outcome is determined. All analysis performed below is based on the pre-registered methods proposed in the first version. The pre-registered article can be found at https://researchers.one/articles/20.10.00004.