PREDICTION MARKET ACCURACY: THE IMPACT OF SIZE, INCENTIVES, CONTEXT AND INTERPRETATION

Article Sidebar

Requires Subscription or Fee PDF (GBP 25)
Published: Dec 19, 2012
DOI: https://doi.org/10.5750/jpm.v6i2.500
Keywords:
prediction market perception market decision support

Main Article Content

Patrick McHugh
Brown University
Aaron L Jackson

Abstract

Enterprises desiring to utilize prediction markets for decision support must consider numerous design factors for their market deployments. Through logistic regression analyses of more than 350 real and play-money prediction markets, this paper evaluates several design issues in order to identify conditions under which prediction markets can effectively contribute to an enterprise decision support process. Two of these design considerations include the size of the trader pool and the nature of trader incentives.  We find that varying the number of market traders has minimal accuracy impact for markets exceeding 10-20 traders and that the impact of financial incentives is contextual and beneficial to market accuracy.  When to act upon market results and at what levels of market support must also be considered.  Our data shows that acting on market output up to three weeks prior to an event’s occurrence and requiring markets to sustain desired price levels for up to 3 weeks before responding to the market signal did not statistically impact the market’s ability to accurately predict an event’s occurrence.  Adjusting the price threshold for market recommendation acceptance to levels between $55 and $80 ($45 and $20) also does not negatively impact market accuracy.  A related measure capturing market uncertainty was found to be the leading predictor of a market’s failure to provide accurate predictions.

Article Details

Issue
Vol. 6 No. 2 (2012)
Section
Articles

References

Anderson, M. J., & Sunder, S. 1995. Professional Traders as Intuitive Bayesians. Organizational Behavior and Human Decision Processes, 64(2): 185-202.

Arendt, L., Priem, R., & Ndofor, H. A. 2005. A CEO-Advisor Model of Strategic Decision Making. Journal of Management, 31(5): 680-699.

Bagehot, W. 1971. The only game in town. Financial Analysts Journal(March-April).

Baldridge, J. V. 1971. Power and Conflict in the University: Research in the Sociology of Complex Organizations. New York: John Wiley.

Barsch, J., Capozzi, M., & Mendonca, L. 2007. How companies approach innovation. The McKinsey Quarterly.

Beaver, W. H. 1981. Market efficiency. The Accounting Review, 56(January): 23-37.

Bell, T. W. 2002. Gambling for the Good, Trading for the Future: The Legality of Markets in Science Claims. Chapman Law Review, 5(1): 159-180.

Berg, J., Nelson, F., & Rietz, T. 2003. Accuracy and Forecast Standard Error of Prediction Markets: 1-45: University of Iowa.

Borison, A., & Hamm, G. 2010. Prediction Markets: A new tool for strategic decision making. California Management Review, 52(4): 125-141.

Brown, S. L., & Eisenhardt, K. M. 1995. Product Development: Past research, present findings and future directions. Academy of Management Review, 20: 343-378.

Camerer, C. F., & Johnson, E. J. (Eds.). 1991. The Process-Performance Paradox in Expert Judgment: How Can Experts Know So Much and Predict so Badly? New York: Cambridge Press.

Carpenter, M. A., & Fredickson, J. W. 2001. Top Management Teams, Global Strategic Posture, and the Moderating Role of Uncertainty. The Academy of Management Journal, 44(3): 533-545.

Chen, K.-Y., & Plott, C. R. 2002. Information Aggregation Mechanisms: Concept, Design and

Implementation for a Sales Forecasting Problem Working Paper: 1-33.

Christiansen, J. D. 2007. Prediction Markets: Practical Experiments in Small Markets and Behaviours Observed. Journal of Prediction Markets, 1(1): 17-41.

Collins, C., & Clark, K. 2003. Strategic Human Resource Practices, Top Management Team Social Networks, and Firm Performance: the Role of Human Resource Practices in Creating Organizational Competitive Advantage. Academy of Management Journal, 46(6): 740-751.

Cooke, R. 1991. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford: Pxford University Press.

Cowgill, B., Wolfers, J., & Zitzewitz, E. 2008. Using Prediction Markets to Track Information flows: Evidence from Google Working Paper.

Eisenhardt, K. M. 1989. Making fast strategic decisions in high-velocity environments. Academy of Management Journal, 32: 543-577.

Eisenhardt, K. M., & Bourgeois, L. J. 1988. Politics of Strategic Decision Making in High-Velocity environments: Toward a Midrange Theory. The Academy of Management Journal, 31(4): 737-770.

Fama, E. F. 1970. Efficient capital markets: A review of theory and empirical work. Journal of Finance, 25: 383-417.

Fama, E. F. 1991. Efficient capital markets: II. Journal of Finance, 46: 1575-1617.

Finkelstein, S., ;, Hambrock, D. C., & Cannellla, A. 2008. Strategic leadership: Theory and reseach on executives, top management teams, and boards. Oxford, U.K.: Oxford University Press.

Finkelstein, s., & Hambrick, D. C. 1996. Strategic Leadership: top Executives and their Effects on Organizations. St. Paul, MN: West Company.

Forsythe, R., & Lundholm, R. 1990. Information aggrgation in an experimental market. Econometrica, 58(2): 309-347.

Forsythe, R., Reitz, T., & Ross, T. 1999. Wishes, expectations and actions: A survey on price formation in election stock markets. Journal of Economic Behavior & Organization 39(1): 83-110.

Gabarro, J. 1987. The Dynamics of Taking Charge. Cambridge, MA: Harvard Business School Press.

Gneezy, U., & Rustichini, A. 2000. Pay enough or don't pay at all. Quarterly Journal of Economics, 115: 791-811.

Grossman, S. J. 1981. An introduction to the general theory of rational expectations under asymmetric information. Review of Economic Studies, 48: 541-559.

Gruca, T. S., Berg, J. E., & Cipriano, M. 2003. The Effect of Electronic Markets on Forecasts of New Product Success. Information Systems Frontiers, 5(1).

Gruca, T. S., Berg, J. E., & Cipriano, M. 2005. Consensus and Differences of Opinion in Electronic Prediction Markets. Electronic Markets, 15(1): 13-22.

Hagel, J., & Brown, J. 2008. Harrah's New Twist on Prediction Markets. Business Week(December 22).

Hair, J., Black, W., Babin, B., Anderson, R., & Tatham, R. 2006. Multivariate Data Analysis. Upper Saddle River, NJ: Pearson Prentice Hall.

Hambrick, D. C., & Mason, P. A. 1984. Upper echelons: The organization as a reflection of its top managers. Academy of Management Review, 9(2): 193-206.

Hayek, F. A. 1945. The use of knowledg ein society. The American Economic Review, 35: 519-530.

Healy, P. J., Linardi, S., Lowery, J. R., & Ledyard, J. O. 2010. Prediction Markets: Alternative Mechanisms for Complex Environments with Few Traders. Management Science, 56(11): 1977-1996.

Hurwicz, L. (Ed.). 1972. On informationally decentralized systems. Amsterdam: North-Holland.

Isenberg, D. 1986. Thinking and managing: A verbal protocol analysis of managerial problem solving. Academy of Management Journal, 29(4): 775-788.

Jamal, K., & Sunder, S. 1996. Bayesian equilibrium in double auctions populated by biased heuristic traders. Journal of Economic Behavior & Organization, 31(2): 273-291.

James, D., & Issac, R. M. 2000. Asset markets: How they are affected by tournament incentives for individuals. American Economic Review, 90: 995-1004.

Kahneman, D., Tversky, A., & Slovic, P. 1982. Judgment under uncertainty: Heuristics and biases. Cambridge, U.K.: Cambridge Press.

Kambil, A., & Van Heck, E. 2002. Making Markets. Cambridge, MA.

LaComb, C. 2007a. GE's Imagination Market. GE Global Research: 15.

LaComb, C. A. B., Janet; Pan, Qimei. 2007b. The imagination market. Information Systems Frontiers, 9: 245-256.

Leech, N., Barrett, K., & Morgan, G. 2008. SPSS for Intermediate Statistics: Use and Inperpretation. New York: Lawrence Erlbaum Associates.

Lucas, R. E. 1972. Expectations and the neutrality of money. Journal of Economic Theory, 4: 103-124.

Luckner, S., & Weinhardt, C. 2007. How to pay traders in information markets? Results from a field experiment. Journal of Prediction Markets, 1(2): 147-156.

Lundholm, R. J. 1991. What affects the efficiency of the market? Some answers from the laboratory. Accounting Review, 66: 468-515.

Malone, T., Laubacher, R., & Dellarocas, C. 2010. The Collective Intelligence Genome. MIT Sloan Management Review, 51(3): 21-31.

McArthur, L. Z. 1976. The lesser influence of consensus than distinctiveness information on causal attributions: A test of the person-thing hypothesis. Journal of Personality and Social Psychology, 33(6): 733-742.

McDonald, M. L., & Westphal, J. D. 2003. Getting by with advice from their friends: CEOs' advice networks and firm's strategic responses to poor performance. Administrative Science Quarterly, 48(1): 1-32.

Mintzberg, H., Raisinghani, D., & Theoret, A. 1976. The structure of unstructured decision processes. Administrative Science Quarterly, 21: 246-275.

Nisbett, R., & Ross, L. 1980. Human Inference: strategies and shortcomings of social judgment. Englewood Cliffs, N.J.: Prentice- Hall.

Ow, T., & Morris, J. G. 2010. An experimental study of executive decision-making with implications for decision support. Journal of Organizational Computing and Electronic Commerce, 20: 370-397.

Peduzzi, P., Concata, J., Kemper, E., Holford, T., & Feinstein, A. 1996. A simulation study of the number of events per variable in loggistic regression analysis. Journal of Clinical Epidemiology, 49: 1373-1379.

Pennock, D., Lawrence, S., Giles, L., & Nielsen, F. 2001. The power of play: Efficiency and forecast accuracy in web market games. Paper presented at the Institute for Operations Research and the Management Sciences National Meeting, Miami, FL.

Pettigrew, A. M. 1985. The awakening giant: Continuity and change in Imperial Chemical Industries. New York: Blackwell.

Plott, C. R. 2000. Markets as Information Gathering Tools. Southern Economic Journal, 67(1): 1-15.

Plott, C. R., & Sunder, S. 1988. Rational Expectations and the Aggregation of Diverse Information in Laboratory Security Markets. Econometrica, 56(5): 1085-1118.

Rosenbloom, E., & Notz, W. 2006. Statistical tests of real-money versus play-money prediciton markets. Electronic Markets, 16(1): 63-69.

Servan-Schreiber, E. W., Justin; Pennock, David M.; Galebach, Brian. 2004. Prediction Markets: Does Money Matter? Electronic Markets, 14(3): 243-251.

Slamka, C., Lickner, S., Seeman, T., & Schroder, J. 2008. An empirical investigation of the forecast accuracy of play-money prediction markets and professional betting markets, 16th European Conference on Information Systems. Galway, Ireland.

Slamka, C., Soukhoroukova, A., & Spann, M. 2008. Event Studies in Real- and Play-Mondey Prediction Markets. The Journal of Prediction Markets, 2(2): 53-70.

Sniezek, J. A., & Van Swol, L. M. 2001. Trust, confidence, and expertise in a judge-advisor system. Organizational Behavior & Human Decision Processes, 84(2): 288-307.

Spann, M., & Skiera, B. 2003. Internet-based virtual stock markets for business forecasting. Management Science, 49(10): 1310-1326.

Spears, B., LaComb, C., Interrante, J., Barnett, J., & Senturk-Dogonaksoy, D. 2009. Examining Trader Behavior in Idea Markets: An Implementation of GE's Imagination Markets. The Journal of Prediction Markets, 3(1): 17-39.

Starbuck, W. H., & Milliken, F. J. (Eds.). 1988. Executive perceptual filters: What they notice and how they make sense Greenwich, CT: JAI Press.

Sunstein, C. 2006a. When crowds aren't wise. Harvard Business Review(September): 20-21.

Sunstein, C. R. (Ed.). 2006b. Deliberation and information markets. Washington D.C.: AEI-Brookings Press.

Surowiecki, J. 2005. The Wisdom of Crowds: Random House.

Taleb, N. N. 2007. The Black Swan: The Impact of the Highly Improbable. New York Random House.

Tversky, A., & Kahneman, D. 1974. Judgment under Uncertainty: Heuristics and Biases. Science(September 27): 1124-1131.

Wolfers, J. 2004. Prediction markets. Journal of Economic Perspectives 18(2): 107-127.

Wolfers, J., & Zitzewitz, E. 2005. Five Open Questions About Prediction Markets. working Paper: 24.

Wolfers, J., & Zitzewitz, E. 2006. Prediction Markets in Theory and Practice. Working Papers (Faculty) -- Stanford Graduate School of Business: 1-14.

Wolk, L. 2008. Real- and Play- money in Prediction Markets: A Time Series Approach. Universiteit Maastricht, Maastricht.