Y2K Bibliography of Experimental Economics and Social Science
Reinforcement Learning

Charles A. Holt, cah2k@virginia.edu, suggestions and corrections welcome
(for online and personal use only)


Blume, Andreas, Douglas DeJong, George R. Neumann, and N.E. Savin (1998) “Learning in Sender-Receiver Games,” University of Iowa, Discussion Paper, presented at the Summer 1998 ESA Meetings. Keywords: experiments, game theory, signaling games, belief learning, reinforcement learning, econometrics. Abstract: This paper compares the explanatory power of belief and reinforcement learning models in the context of signaling games. Email Contact: ablume@blue.weeg.uiowa.edu

Blume, Andreas, D. V. DeJong, G. R. Neumann, and N. E. Savin (1999) “Inferring Learning Rules from Experimental Game Data,” University of Iowa, Discussion Paper, presented at the Summer 1999 ESA Meeting. Keywords: experiments, methodology, econometrics, belief learning, reinforcement learning, Monte Carlo study. Abstract: The paper reports simulations of behavior using either reinforcement or ficticious play learning rules. When the wrong model is used to obtain parameter estimates, the fit is typically quite good, even for medium-sized samples, and tests of non-nested models may provide an incorrect conclusion about which model generated the data. Results are better for tests based on general formulations that include both learning rules as special cases. Email Contact: ablume@blue.weeg.uiowa.edu

Bouchez, Nichole Marie (1999) “Learning Models in a Three by Three Bimatrix World,” University of California at Santa Cruz, Discussion Paper, presented at the Summer 1999 ESA Meeting. Keywords: experiments, game theory, learning, 3x3 matrix games, belief learning, reinforcement learning. Email Contact: bouchez@cats.ucsc.edu

Bracht, Juergen, Christian Lebiere, and Dieter Wallach (1998) “A Comparison of ACT-R and Reinforcement Based Learning in Experimental Games with Unique Mixed Strategy Equilibria,” University of Basel, Discussion Paper, presented at the Summer 1998 ESA Meeting. Keywords: experiments, game theory, learning, reinforcement learning, ACT-R. Abstract: An ACT-R theory is applied to strategic behavior in normal form games with unique mixed strategies. The simulations converge to the Nash equilibrium and follow adjustment observed patterns in experimental data. Reinforcement learning models are discussed. Email Contact: wallachd@ubaclu.unibas.ch

Bush, Robert R., and Frederick Mosteller (1955) Stochastic Models for Learning*, New York: Wiley. Keywords: experiments, decisions, learning, stochastic learning, reinforcement learning. Abstract: This is a classic reference on psychological theories of learning.

Camerer, Colin F., and Teck-Hua Ho (1999) “Experience Weighted Attraction Learning in Normal-Form Games,” Econometrica, 67827-874. Keywords: experiments, game theory, learning, reinforcement learning. Abstract: This paper presents a general model of learning that nests the standard belief and reinforcement learning specifications. Many extensions are discussed and incorporated, and the estimated parameters support the use of a hybrid model. Email Contact: camerer@hss.caltech.edu

Chen, Yan (1998) “Asynchronicity and Learning in Cost Sharing Mechanisms,” University of Michigan, Discussion Paper, presented at the Fall 1998 ESA Meetings. Keywords: experiments, public goods, mechanisms, cost-sharing, average cost pricing mechanism, serial pricing mechanism, learning, reinforcement learning. Abstract: This paper reports an a comparison of serial and average-cost pricing mechanisms. Performance is similar under complete information, but the serial mechanism performs better under limited information. Convergence to the Nash equilibrium is faster than predicted by a reinforcement learning model; other learning models are considered. Email Contact: yanchen@umich.edu

Cooper, David, Nicholas Feltovich, Alvin Roth, and Rami Zwick (1998) “Learning in Ultimatum Games,” University of Pittsburgh, Discussion Paper, presented at the Summer 1998 ESA Meetings. Keywords: experiments, bargaining, ultimatum games, reinforcement learning, fairness, inequality aversion. Abstract: Changes in behavior over time in ultimatum games lead the authors to consider learning models. The simple reinforcement model does not provide a good explanation of dynamic patterns unless reciprocity and autocorrelation factors are incorporated. Email Contact: djc13@guinness.som.cwru.edu

Erev, Ido, and Alvin E. Roth (1998) “Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria,” American Economic Review, 88:4 (September), 848-881. Keywords: experiments, game theory, learning, reinforcement learning. Abstract: Models of reinforcement learning are used to explain dynamic behavior patterns in previously published experiments with unique mixed strategy equilibria. The simplest one-parameter learning model (with a parameter estimated from other experiments) produces simulations that outperform Nash predictions. The fit is improved by adding parameters that represent "forgetting" and "experimentation." Email Contact: erev@techunix.technion.ac.il

Erev, Ido, and Amnon Rapoport (1998) “Coordination, "Magic," and Reinforcement Learning in a Market Entry Game,” Games and Economic Behavior, 23:2 (May), 146-175. Keywords: experiments, game theory, entry game, coordination, reinforcement learning. Email Contact: erev@techunix.technion.ac.il

Erev, Ido, Sharon Gilat, Galia Shabtai, and Doron Sonsino (1998) “On the Likelihood of Repeated Betting: A Study of the Robustness and Validity of Geanakoplos and Sebenius No-Betting Conjecture,” Technion - Israel Institute of Technology, Discussion Paper, presented at the Summer 1998 ESA Meeting. Keywords: experiments, decisions, betting, no-betting conjecture, reinforcement learning. Abstract: The experiment places subjects in a setting where standard theory predicts that players will avoid betting on zero-sum outcomes. The observed incidence of betting behavior is high, even after many repetitions. Time trends in decisions are explained by a variant of a reinforcement-learning model. Email Contact: sonsino@ie.technion.ac.il

Erev, Ido, and Alvin E. Roth (1999) “On the Role of Reinforcement Learning in Experimental Games; The Cognitive Game Theory Approach,” in Games and Human Behavior: Essays in Honor of Amnon Rapoport, edited by D. Budescu, I. Erev and R. Zwick, Erlbaum, 53-77. Keywords: experiments, game theory, reinforcement learning. Email Contact: aroth@hbs.edu

Erev, Ido, Yoella Bereby-Meyer, and Alvin E. Roth (1999) “The Effect of Adding a Constant to All Payoffs: Experimental Investigation, and a Reinforcement Learning Model with Self-Adjusting Speed of Learning,” Journal of Economic Behavior and Organization, 39:1 (May), 111-128. Keywords: experiments, game theory, reinforcement learning, incentives. Abstract: The paper reports a nonlinear effect of adding a constant to all payoffs, and discusses the implications for reinforcement learning models. Email Contact: aroth@hbs.edu

Grosskopf, Brit (1998) “Competition, Aspiration and Learning in the Ultimatum Game: An Experimental Investigation,” Universitat Pompeu Fabra, Discussion Paper presented at the 1999 European Economics Association Meetings. Keywords: experiments, bargaining, ultimatum games, multiple responders, learning, reinforcement learning, virtual learning, order-of-treatment effects, experimental design. Abstract: Demands are higher in bilateral ultimatum game experiments than in competitive games with one proposer and three responders. The paper proposes a modification of standard reinforcement learning models to allow for virtual learning, i.e. reinforcement of unchosen strategies. Email Contact: grosskop@upf.es

Roth, Alvin E., and Ido Erev (1995) “Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term,” Games and Economic Behavior, 8:1 (January), 164-212. Keywords: experiments, game theory, learning, reinforcement learning. Email Contact: aroth@hbs.edu

Stahl, Dale (1999) “A Horse Race Among Reinforcement Learning Models,” University of Texas, Discussion Paper, presented at the Spring 1999 Public Choice Meetings. Keywords: experiments, game theory, learning. Email Contact: stahl@eco.utexas.edu

Swarthout, Todd, and Mark Walker (1999) “Reinforcement, Belief Learning, and Information Processing,” University of Arizona, Discussion Paper, presented at the Summer 1999 ESA Meeting. Keywords: experiments, game theory, belief learning, reinforcement learning. Email Contact: swarthout@nt.econlab.arizona.edu