Open Access Open Access  Restricted Access Subscription or Fee Access

A comparison of forecasting methods: fundamentals, polling, prediction markets, and experts

Deepak Pathak, David Rothschild, Miroslav Dudik

Abstract


We compare Oscar forecasts derived from four data types (fundamentals, polling, prediction markets, and domain experts) across three attributes (accuracy, timeliness and cost effectiveness). Fundamentals-based forecasts are relatively expensive to construct, an attribute the academic literature frequently ignores, and update slowly over time, constraining their accuracy. However, fundamentals provide valuable insights into the relationship between key indicators for nominated movies and their chances of victory. For instance, we find that the performance in other awards shows is highly predictive of the Oscar victory whereas box office results are not. Polling- based forecasts have the potential to be both accurate and timely. Timeliness requires incentives for frequent responses by high-information users. Accuracy is achieved by a proper transformation of raw polls. Prediction market prices are accurate forecasts, but can be further improved by simple transformations of raw prices, yielding the most accurate forecasts in our study. Expert forecasts exhibit some characteristics of fundamental models, but are generally not comparatively accurate or timely. This study is unique in both comparing and aggregating four traditional data sources, and considering critical attributes beyond accuracy. We believe that the results of this study generalize to many other domains.

Keywords


prediction; Oscars; regression; probabilities; translation; combining forecasts

Full Text:

PDF

References


Arrow, K. J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., Ledyard, J. O., Levmore, S., Litan, R., Milgrom, P., Nelson, F. D., et al., 2008. The promise of prediction markets. Science 320 (5878), 877.

Clemen, R. T., Winkler, R. L., 1986. Combining economic forecasts. Journal of Business & Economic Statistics 4 (1), 39-46.

Diebold, F. X., Mariano, R. S., 2002. Comparing predictive accuracy. Journal of Business & Economic Statistics 20 (1).

Erikson, R. S., Wlezien, C., 2008. Are political markets really superior to polls as election predictors? Public Opinion Quarterly 72 (2), 190-215.

Fair, R., 2011. Predicting presidential elections and other things. Stanford University Press.

Ghitza, Y., Gelman, A., 2013. Deep interactions with MRP: Election turnout and voting patterns among small electoral subgroups. American Journal of Political Science.

Goel, S., Reeves, D. M., Watts, D. J., Pennock, D. M., 2010. Prediction without markets. In: Proceedings of the 11th ACM conference on Electronic commerce. ACM, pp. 357-366.

Granberg, D., Brent, E., 1983. When prophecy bends: The preference-expectation link in U.S. presidential elections. Journal of Personality and Social Psychology 45 (3), 477-491.

Granger, C. W., Ramanathan, R., 1984. Improved methods of combining forecasts. Journal of Forecasting 3 (2), 197-204.

Guedj, O., Bouchaud, J.-P., 2005. Experts' earning forecasts: Bias, herding and gossamer information. International Journal of Theoretical and Applied Finance 8 (7), 933-946.

Harvey, D. S., Leybourne, S. J., Newbold, P., 1998. Tests for forecast encompassing. Journal of Business & Economic Statistics 16 (2), 254-259.

Hong, H., Kubik, J. D., Solomon, A., 2000. Security analysts' career concerns and herding of earnings forecasts. The Rand Journal of Economics, 121-144.

Hummel, P., Rothschild, D., 2014. Fundamental models for forecasting elections at the state level. Electoral Studies 35, 123-139.

Lock, K., Gelman, A., 2010. Bayesian combination of state polls and election forecasts. Political Analysis 18 (3), 337-348.

Manski, C. F., 2006. Interpreting the predictions of prediction markets. Economics Letters 91 (3), 425-429.

Pennock, D. M., Lawrence, S., Giles, C. L., Nielsen, F. A., et al., 2001. The real power of artificial markets. Science 291 (5506), 987-988.

Rothschild, D., 2009. Forecasting elections comparing prediction markets, polls, and their biases. Public Opinion Quarterly 73 (5), 895-916.

Rothschild, D., 2015. Combining forecasts for elections: Accurate, relevant, and timely. International Journal of Forecasting 31, 952-964.

Rothschild, D., Wolfers, J., 2011. Forecasting elections: Voter intentions versus expectations. Available at SSRN: http://ssrn.com/abstract=1884644.

Squire, P., 1988. Why the 1936 Literary Digest poll failed. Public Opinion Quarterly 52 (1), 125-133.

Tibshirani, R., 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 267-288.

Wang, W., Rothschild, D., Goel, S., Gelman, A., 2015. Forecasting elections with non-representative polls. International Journal of Forecasting 31, 980-991.

Wolfers, J., Zitzewitz, E., 2004. Prediction markets. Tech. rep., National Bureau of Economic Research.

Wolfers, J., Zitzewitz, E., 2006. Interpreting prediction market prices as probabilities. Tech. rep., National Bureau of Economic Research




DOI: http://dx.doi.org/10.5750/jpm.v9i2.1048

Refbacks

  • There are currently no refbacks.