Charlie Holt's
Homepage

Economics
Department

Classroom
Games

Experimental
Economics

Charles Holt's Research Interests

Narrative Description of Research on Stochastic Game Theory

Academic Vita   (with links to *.pdf files for all publications)

Data and Instructions  (appendices for laboratory experiments)

Y2K Bibliography  (2000 listings in experimental economics) 

Survey Papers on Experimental Economics (by topic)
 

Relaxing the Assumptions of Perfect Rationality
    I am particularly interested in decision making in interactive "games" when people are not assumed to be perfectly rational.  This research is based on mathematical models of strategic behavior in which the best decision depends on what one thinks other(s) will do, e.g. what they will bid in an auction.  These beliefs can be determined by learning, or in the absence of prior experience, by introspection.  Beliefs determine the payoffs anticipated for each decision, and payoffs determine the choice probabilities.  Even small amounts of "noise" in decision making can have a large "snowball" effect in an interactive game.  In some contexts, these models of bounded rationality predict that behavior will be essentially the reverse of what it would be in the case with perfect rationality.  Some of these papers are listed on my Vita and pertain to coordination, social dilemmas, rent seeking, public goods, games played only once, etc. 

Laboratory Experiments in Economics
     The new theories are tested in laboratory experiments by letting students make decisions in markets and games in which their earnings depend on the decisions made by themselves and others.  For example, the profitability of a particular price may depend on price itself and on the prices charged by rival sellers.  The observed choice data usually reject the hypothesis of perfect rationality that is typically used in economic theory, and the models of bounded rationality are used to explain and predict anomalies.  For a summary of this "stochastic game theory" that is written for a broad scientific audience, see: "Stochastic Game Theory: for Playing Games, Not Just for Doing Theory" (with Jacob Goeree), in the September 1999 Proceedings of the National Academy of  Sciences.

Experimental Economics 
     The Y2K Bibliography of Experimental Economics and Social Science lists about 2000 publications and over 500 working papers, with selected abstracts and keywords that have been used to construct special reference lists.  Related work on experimental economics can be found in the new Kluwer Academic Press Journal, Experimental Economics that I co-edit.  For a general introduction to experimental economics, see the 1993 Princeton University Press monograph Experimental Economics [Contents] (with Doug Davis).  The cover for this book is shown in the box at the upper right part of this page.   This research is also used to devise interactive classroom games for use in teaching economics at all levels (click on Classroom Games). 

Narrative Description of Recent Research: 
Stochastic Game Theory and Human Behavior
          National Science Foundation Grants 
          SES 0094800 (PI, with Jacob Goeree and 8 co-PIs at 8 universities)
          SBR 9818683 (PIs: Jacob Goeree and Charles Holt) 
          SBR 9617784 (PIs: Simon Anderson, Jacob Goeree, and Charles Holt) 

Contents:
I. Game Theory and Laboratory Experiments 
II. Three Complementary Approaches 
        Introspection and Games Played Once 
        Equilibrium with Bounded Rationality
        Learning and Evolution 
III. Applications of Stochastic Game Theory 
        Rent Seeking 
        Coordination 
        Binary Choice Games 
IV. Incorporating a Broader Range of Human Motivation 
        Public Goods and Altruism 
        Bargaining and Fairness 
        Risk Aversion in Lottery Choice, Games, and Auctions 
V. Summary of Major Scientific Insights 
         Noisy Introspection: A Model of Disequilibrium Behavior for One-Shot Games
         Quantal Response Equilibrium Analysis for Games with Continuous Strategies
         Stochastic Equilibrium Models with Altruism, Inequity Aversion, and Risk Aversion
         Stochastic Learning Equilibrium: Long Run Consequences of Learning
         Stochastic Potential
VI. Abstracts of Research Papers and Publications 
 

I. Game Theory and Laboratory Experiments: Historical Perspective

        Economics in the early twentieth century pertained almost exclusively to "price theory," which assumes that traders respond passively to price signals without thinking strategically about rivals' actions and reactions. This theory can be effectively applied to `thick' markets with lots of traders and good information, but John von Neumann and Oscar Morgenstern realized that many economic interactions like bilateral bargaining or bidding in auctions are not adequately modeled by price theory. Although their Theory of Games and Economic Behavior was reviewed on the front page of the New York Times and was widely heralded in the profession, John Nash, a young mathematics graduate student at Princeton, noticed a flaw. The only games that were "solved" were zero-sum games, i.e. those with the special property that one person's gain exactly equals another's loss. Nash approached von Neumann, who was then Chair of the Princeton Mathematics Department, with a more general equilibrium analysis, which von Neumann dismissed as merely a mathematical "fixed point" argument. 

        The notion of equilibrium is analogous to a state of rest in a physical system. An equilibrium in an interactive social system is, roughly speaking, a state in which people choose strategies of action, and there is no tendency for the system to change if each person does not want to change their own strategy, given what they know about the strategies chosen by the others. Nash provided a formal definition of equilibrium and proved that such an equilibrium would in fact exist in a wide class of strategic games. Nash's thesis advisor, Arnold Tucker, sent the proof to the Proceedings of the National Academy of Sciences about fifty years ago, resulting in a half-page Nobel-Prize winning paper. 

        On the other coast, a group of mathematicians and economists at the RAND Corporation heard about Nash's proof and were immediately interested, since the von Neumann/Morgenstern zero-sum paradigm did not fit well with the possibility of mutual destruction from a nuclear exchange that loomed so large in the 1950's. The same day that word of Nash's proof arrived in Santa Monica, two mathematicians ran a laboratory experiment designed to determine whether behavior of financially motivated people would conform to the predicted behavior (it didn't). Tucker happened to pass by and see the payoff numbers written on the blackboard, and he used them to invent the story of the "prisoner's dilemma" for a talk on recent developments in game theory at the Stanford Psychology Department. 

        The prisoner's dilemma is a game in which the best decision one can make (in private) does not depend on what the other person does, i.e. the prosecutor makes promises and threats to two prisoners in such a manner that each is better off confessing to a crime regardless of whether the other confesses, even though both would be better off if neither confessed. The Nash equilibrium is for both to confess, which produces an (admittedly unhappy) state of rest. 

        The prisoner's dilemma has become a paradigm for many aspects of strategic interaction, although there are others that will be discussed below. In the meantime, game theory itself has finally assumed the central role in economics first envisioned by von Neumann and Morgenstern, although ironically, the Nash equilibrium has replaced their earlier solutions as the central element in the theory. Game theory is being increasingly applied in law, management, psychology, political science, and to some extent in systems engineering, artificial intelligence, biology, sociology and anthropology. In my opinion, game theory is the closest thing there is to a unified theory of social science. 

        Game theory is especially useful in situations where well developed markets and prices are not available, e.g. for the analysis of coordination within a firm, family, tribe, or government agency. It is also essential in contexts where people are trying to anticipate others' decisions, as with firms in concentrated industries, or governments considering strategic international trade issues. To a large extent, the focus of economic analysis is on the equilibria of complex interactions, and the requirements for formal models of strategic behavior limit the behavior that can be accommodated. Although the Nash equilibrium can apply to a wide range of social situations, for example, it still requires very restrictive rationality assumptions. An alternative approach that emerges from computer science and artificial intelligence is to write complex computer simulations that program adaptive responses into simulated human players. A second alternative is to rely more on non-mathematical insights about human behavior gleaned from the social psychology literature. Our work is influenced by these approaches; we seek to broaden the behavioral possibilities that can be considered, while preserving the possibility of using formal models to produce precise predictions that increase the power and usefulness of game theory. The next section describes three closely related perspectives of this new stochastic game theory: introspection for games played once, learning for repeated games, and equilibrium for long-run steady states. 
 

II. Three Complementary Approaches

Introspection and One-Shot Games
        The increasing adoption of game-theoretic models in law, business, and social sciences, is problematical in light of its failures to explain human decisions in many laboratory settings. "Ten Little Treasures of Game Theory and Ten Intuitive Contradictions" (Goeree and Holt, American Economic Review, December 2001) is intended to serve as a wake-up call for those who are not familiar with earlier experiments by economists and psychologists. Each of the games considered has one set of parameters (the "treasure treatment") for which the Nash equilibrium predicts very well, but a change in a payoff parameter that does not alter the Nash equilibrium causes a large shift in observed behavior in a parallel "contradiction treatment." Moreover, these contradictions of theory are intuitive.  For a popular-press account of this paper, see Hal Varian's article:  "What, Exactly, Was on John Nash's Beautiful Mind?" New York Times, April 11, 2002). 

        For example, consider a "matching pennies" game in which a soccer player can either kick to the East side or the West side, and the goalie can either dive East or West. Suppose that a "match" (the goalie dives to meet the kick) gives payoffs of 1 to the goalie and -1 to the kicker, whereas a mismatch gives -1 to the goalie and +1 to the kicker. Each player wants to be unpredictable, and the only Nash equilibrium for this symmetric game is for each player to choose each direction with probability 1/2 (otherwise at least one player would want to change their behavior). This prediction is confirmed in laboratory matching pennies games. Now consider what happens if the fans of the goalie's team, who are seated on the East side of the field, announce publicly that they will donate ten thousand dollars to the team scholarship fund if the goalie blocks a kick on the East side. This only increases the goalie's payoff for the (dive-East, kick-East) outcome, but it cannot increase the Nash equilibrium probability that the goalie dives East, since the kicker must remain indifferent over the two kick directions in a Nash equilibrium with randomized decisions. This unintuitive prediction, that a change in a player's own payoff for one of the outcomes will not change that player's probability of choosing that outcome, is dramatically rejected in the experiments based on asymmetric matching pennies games. Thus the Nash equilibrium works well in the symmetric game and it predicts poorly in the asymmetric matching pennies game. 

        We show that intuitive contradictions like this are explained by a simple model of "noisy introspection" (about what the other person might do, of what the other person thinks I might do, of what they think I think, etc.). It is well known that a succession of noise-free iterated best responses may not converge, or if it does, it will converge to a Nash equilibrium, which cannot explain behavior in the "contradiction" treatments. The innovation in our model is the injection of increasing amounts of noise into the iterated thought process: one is assumed to be less uncertain about the other person's decisions than about the other's beliefs about my decisions, etc. In "A Model of Noisy Introspection," (Goeree and Holt, forthcoming in Games and Economic Behavior) we show that the introspective equilibrium always exists and that it provides a good explanation of deviations from standard equilibrium predictions in a series of 37 one-shot game experiments. This model relaxes two extreme rationality assumptions of a Nash equilibrium: rational choice (no mistakes in choosing the action with the highest expected payoff) and rational expectations (belief distributions about others' actions correspond to actual distributions of decisions). This noisy introspection model is useful for the analysis of games played only once, which is a good description of many interactions in politics, war, legal disputes, and auctions. 

Logit Equilibrium
        Expectations will only be accurate by coincidence for a game played once, but with repetition, players may come to learn what to expect, and beliefs will more closely approximate the distributions of actual decisions. We use learning models to explain patterns of adjustment in some of the experimental papers, but when behavior stabilizes and expectations become more accurate, the use of an equilibrium model is appropriate. The logit equilibrium imposes the rational expectations assumption, but relaxes the rational choice assumption by letting the probability of making the best decision be is an increasing function of the difference in expected payoffs, just as the probability of judging correctly which light is brightest is positively related to the true difference in brightness. This is essentially the notion of "probabilistic choice" (due to the psychologist, Duncan Luce, and others) injected into the equilibrium analysis of a game. (The "logit" equilibrium is obtained when the popular logit probabilistic choice rule is used, which causes choice probabilities to be proportional to exponential functions of expected payoffs.) In order to relate these concepts, think of a Nash equilibrium as a kind of economists' heaven (depending on your perspective) where everyone is perfectly rational and knows that the others are too. Then the logit equilibrium is like a situation where people make mistakes but have learned to anticipate other's mistakes, and the noisy introspection model applies to a "one-shot" encounter before this learning has taken place. 

        The effects of bounded rationality are illustrated dramatically in "Anomalous Behavior in a Traveler's Dilemma?" (Capra, Goeree, Gomez, and Holt, American Economic Review, 1999). The motivating story is that two travelers lose bags with identical value, and the airline representative asks them to fill out claims independently, with the constraint that claims must be in some pre-specified range, e.g. between $80 and $200. The travelers will be reimbursed if the claims are equal, but if claims are unequal the assumption is that the high claimant overstated the value, so both are reimbursed at the minimum of the two claims. Moreover, the low claimant receives a reward of $R and the high claimant's reimbursement is reduced by a corresponding amount. For example, if the claims are 99 and 100, then the minimum claim is 99, so the first person receives 99 + R and the second earns 99 - R. Each person would have an incentive to "undercut" the other's claim if it were known, and the unique Nash equilibrium (with rational choice and rational expectations) is for both to claim the lowest possible amount ($80 in the example). Notice that this argument does not depend on the size of the penalty/reward rate R. i.e. whether it is a matter or pennies or $100. 

        The traveler's dilemma is no more about travel claims than the prisoner's dilemma is about confessions. In each case, the goal is to devise a stylized situation or paradigm that captures the key elements of the strategic landscape for a class of applications. This means preserving the essential economic structure of the game and stripping away or holding constant the rich framing and contextual elements of any particular choice problem, so that the effects of changes in economic incentives can be evaluated. As noted above, the remarkable feature of the traveler's dilemma game is that the equilibrium prediction is independent of R, the penalty reward parameter. But if R is low (e.g. pennies), there is little risk in raising the travel claim, and if R is large (e.g. $50), there is considerable risk, so intuition suggests that claims will be higher with a low penalty parameter. This intuition is confirmed in the laboratory experiments: the Nash equilibrium predicts well for high penalty parameters, but the data cluster at the opposite end of the set of feasible claims when R is low. The logit equilibrium model explains data in both cases, at least after the data settle down in a steady state. 

Learning and Evolution
        If you look at the mathematical expressions in the traveler's dilemma paper, you will probably be skeptical about whether such abstract formulas can actually predict the behavior of human subjects who typically rely on intuition instead of pencil-and-paper calculations. People in the experiments adapt and learn, if one sees high claims in the first rounds of the experiment, it is reasonable to expect high claims in subsequent rounds. We use models of "belief" learning with noisy decisions to explain the pattern of adjustments to equilibrium. In "Stochastic Game Theory: For Playing Games, Not Just for Doing Theory," (Holt an Goeree, Proceedings of the National Academy of Sciences, 1999), we use computer simulations of randomly matched players to predict both the directions and the steady state levels of the adjustment paths observed in experiments. If the simulated learning process has enough memory to learn the exact probability distribution of claims (e.g. "fictitious play"), then simulations show that behavior converges to the logit equilibrium. A more refined analysis of learning is presented in "An Experimental Study of Imperfect Price Competition," (Capra, Goeree, Gomez, and Holt, International Economic Review, 2002) where we show that the steady state of the learning process is the logit equilibrium under fictitious play, but that small differences persist when beliefs are more responsive to recent experience. This paper introduces the idea of a stochastic learning equilibrium, which is close to the logit equilibrium for the applications we have considered, but which can differ from the logit equilibrium when learning is subject to "recency effects". 

        One reaction that we sometimes encounter is that noise is transitory, just as random wave action may not have much effect on the predictable patterns of ocean tides. This reaction is understandable, since one might expect a small amount of decision error would only add "noise" around the predictions of a theory with no errors, e.g. the predicted Nash equilibrium claim of 80 in the traveler's dilemma example. In an interactive game, however, there can be synergies that make the effects of noise much more analogous to the momentum that accumulates in atmospheric weather patterns. For example, if noise causes an upward drift in one person's decisions, then this changes the landscape of others' expected payoffs, causing them to drift upward, and the resulting "herd effect" may move the data to a noisy equilibrium that is nowhere near the Nash equilibrium prediction. In the process, the boundedly rational players are not solving any equations, they are merely reacting to the positive and negative results of noisy local changes. We use an interactive version of the Fokker-Planck equation from theoretical physics to model a naive "hill-climbing" adjustment process (with Brownian motion) that is shown to have the logit equilibrium as a steady state. This work is reported in "Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning" (Anderson, Goeree, and Holt). 

        The recent history of economic methodology supports the Lakatosian notion that a theory will not be abandoned unless a better alternative is available. In fact, laboratory violations of Nash predictions have had little effect on applied game theory models in the absence of a widely accepted alternative. If anything, game theory has become more enshrined in its mathematical armor, with new specialty journals, recent Nobel prizes, and an international Game Theory Society holding its first World Congress in July 2000. The isolation of game theory is reflected in a remark by Nobel Laurate Reinhard Selten: "Game theory is for proving theorems, not for playing games." One problem has been that strategic interactions occur in uncontrolled situations that are subject to various alternative interpretations. Controlled experiments have been relatively new in mainstream economics, and until recently experimental economics has not been taught at the top-10 U.S.graduate programs. Consequently, game theory has undergone considerable refinement with only indirect exposure to behavioral evidence, and indeed the adjectives that precede some theoretical concepts sound more theological than scientific, e.g. "subgame perfectness," "the divinity refinement," or "purification of equilibria." Sometimes theoretical intuition even seems to substitute for data; as Richard Thaler once quipped at a psychology conference, "When economists say that the evidence is mixed, they often mean that the theory says one thing and the data say something else." The goal of the traveller's dilemma paper and the others in the bounded rationality series is to redirect the focus to explaining intuitive behavioral anomalies with formal models that relax the perfect rationality assumptions. The next section describes a number of applications where intuitive deviations from Nash predictions are explained by the incorporation of noise into interactive models. 
 

III. Applications of Stochastic Game Theory

Rent Seeking
        Economists are typically thought of as being apologists for the economic system, by providing a theology that explains how economic interactions produce efficient outcomes via an "invisible hand." Economics experiments largely support Adam Smith's remarkable insights about market efficiency, but economists have been quick to spot inefficiencies arising from imperfections and non-market interactions. Perhaps the most perverse example occurs when an attempt to give away an indivisible "good" results in a net loss to the group of potential recipients as a whole. An example was provided to me in a recent letter from a colleague: 

"...say that a franchise was going to be awarded to a company by a State government. Once the franchise is given out, the company getting it would be a monopolist. Obviously, many companies would like to be awarded such a contract and would spend resources on lawyers, lobbyists, and politicians trying to get it. When the `competition' is over, however, there is only one winner who gets the franchise and earns the `rents' available....For the losers, all of the resources they expended are lost....The conventional theory for such auctions exhibits several features. For example, in equilibrium, all rent is dissipated by the expenditures of firms trying to get the franchise.... not exactly what we observe in reality. In many cases it may be possible for firms to over-dissipate the rents by collectively spending more to receive a franchise than the franchise is worth to any one of them (a clear social inefficiency)." 

This rent seeking process is like an "all-pay auction" in which the prize goes to the highest bidder, but all have to pay their bids, since bidding in done by spending money on lobbying and franchise application procedures. A Nash equilibrium for these rent-seeking games can produce under-dissipation or exact dissipation of rents, but never over-dissipation, since this would yield losses that could be avoided by not competing for the franchise. In contrast, over-dissipation is possible in models that relax the assumption of perfect rationality, as described in "Rent Seeking with Bounded Rationality: An Analysis of the All Pay Auction" (Anderson, Goeree, and Holt, 1998, Journal of Political Economy). Even though rent is always exactly dissipated in a Nash equilibrium for the all-pay auction, the logit equilibrium predicts that rent dissipation is greater when there are more competitors for the rent. This paper provides an explanation for the qualitative patterns of over-dissipation observed in some laboratory experiments. 

Coordination
        Despite the well-known efficiency properties of competitive markets, economists have long been concerned that there may be multiple equilibria in complex social systems. The possibility of becoming mired in a bad equilibrium dates back at least to Thomas Malthus' notion of a "general glut." The problem is one of coordination: e.g. it may not make sense for me to go to the market to trade if you are going to stay home on the farm. Our papers are based on a model of a coordination game in which workers choose effort levels, and the group product is the minimum of the efforts. Thus the output is determined by the weakest link in the production chain. This is analogous to Rousseau's "stag hunt" where the chances of a successful encirclement of the prey depend on the weakest link around the circle of hunters. In the economic application, each worker's payoff is the minimum of the efforts minus the cost of their own effort. The game is specified so that any common effort level is a Nash equilibrium, regardless of the number of people or the cost of effort, since a unilateral increase in effort is costly and does not affect the minimum, and a unilateral decrease reduces the minimum by more than the personal cost saving. Thus the Nash equilibrium makes no prediction about average effort, despite the intuitive argument that successful coordination should be harder with a higher effort cost or with more people (with a large group, it is more likely that somebody will ruin the group output with a low effort). 

        This intuition, that efforts will be higher with low effort cost, is confirmed by our laboratory experiments, and again, the logit equilibrium predictions track the data nicely. Efforts are intermediate and tend to rise in the low-cost treatment, and they start at about the same initial level and tend to fall in the high-cost treatment. In each case, the effort level trajectories tend to flatten after several periods of random rematching of players, and then average levels are quite close to the predictions of the logit equilibrium ("An Experimental Study of Costly Coordination," Goeree and Holt, 1999). 

        The stag hunt game is another of those game-theory paradigms, which in this case has been used as a proving ground for theories designed to choose among multiple Nash equilibria. Our approach predicts a unique probability distribution over decisions, and the derivation of this distribution is analogous to the minimization of potential energy in a physical system. In game theory, a potential function is a mathematical formula that is positively related to individual players' payoffs: when a change in a player's own decision raises that player's payoff, then this change necessarily raises the value of the potential function by the same amount, and vice versa for decreases. If such a potential function exists for the game, then each person trying to increase their own payoff may produce a group result that maximizes the potential function for the game as a whole. Think of two people holding adjacent sides of a treasure box, with one pulling uphill along the East-West direction and the other pulling uphill along the North-South axis. Even though each person is only pulling in one direction, the net effect will be to take the box to the top of the hill, where there is no tendency to change (a Nash equilibrium that maximizes potential). When there is some randomness in the individual directional movements, the dynamic system maximizes a "stochastic potential," which is the expected value of ordinary potential function of the game plus a measure of dispersion (entropy). We show that the maximum of stochastic potential is a logit equilibrium, which produces a unique distribution of effort levels in the coordination game. As effort costs rise, the distribution of efforts decreases, and vice versa, a result that is intuitive but not predicted by a Nash equilibrium. The intuition is that the randomness makes behavior sensitive not just to whether the incline is increasing or decreasing, but to how steep the incline actually is. This research is reported in "Minimum Effort Coordination Games: Stochastic Potential and Logit Equilibrium" (Anderson, Goeree, and Holt, Games and Economic Behavior, 2001). 

Binary Choice Games
        Many games of interest to economists and political scientists involve many players who each make a binary choice, e.g. whether to vote, enter a market, go to a potentially crowded restaurant, volunteer to make a dangerous rescue, or contribute to a public good. Previous experiments have reported seemingly anomalous results, e.g. over-entry relative to Nash predictions in one study and under-entry in another. Another example is the "volunteer's dilemma," in which only one volunteer is needed to provide a public good, but each person would prefer that somebody else incur the cost of volunteering. The Nash equilibrium exhibits an unintuitively strong level of free riding: the probability of getting at least one volunteer is predicted to go to zero as the number of potential volunteers becomes large. This prediction is contradicted by experiments, which show that an increase in the number of potential volunteers increases the probability of getting at least one. These and other anomalies are consistent with the qualitative predictions of a quantal response equilibrium. For example, the same structural conditions that tend to cause over-entry in some market games are also associated with over-participation in some voting game experiments, under both majority and proportional rule procedures. A simple graphical device for predicting behavior in binary choice games is provided in "An Explanation of Anomalous Behavior in Binary-Choice Games: Entry, Voting, Public Goods, and the Volunteer's Dilemma" (Goeree and Holt, 2000). 
 

IV. Incorporating a Broader Range of Human Motivation

Public Goods and Altruism
        Economists' assumptions about selfish preferences generally echo Adam Smith's famous remark that "...it is not for the benevolence of the butcher or the baker...." A number of researchers, however, have explored the implications of other-regarding behavior (altruism, inequity aversion, etc.) in the context of public goods and bargaining models. The idea behind a public goods game can be illustrated by Adam Smith's street light example. The benefits of putting a street light in front of a home may not justify the cost to one person. But if a large number of people who pass by will benefit from the light, then the social benefits exceed the cost, and the light should be provided even though the private benefits to the homeowner do not exceed the cost. The temptation in this situation is for each person to "free ride" and rely on others' lights, which is a rationale for public provision of this service on streets with sufficiently high traffic. However, we do observe many voluntary contributions to public goods and charities, and this raises the issue of how much people are motivated by concerns for others' well being. In "A Theoretical Model of Altruism and Decision Error in Public Goods Games" (Anderson, Goeree, and Holt, 1998, Journal of Public Economics), we show that altruism can be incorporated into a logit equilibrium analysis. The resulting hybrid model produces intuitive predictions, e.g. that contributions will go up if the marginal value of the public good increases, even though it is a Nash equilibrium for (selfish) players to free ride. With altruism, an increase in the number of people who enjoy the public good will increase contributions, which is consistent with some observed "group size" effects. 

        Following the initial experiments of sociologists Marwell and Ames, there is a large literature on the causes of voluntary giving in controlled environments. Our experiment uses a design in which the benefits of one's own contribution to others (the "external return") is varied independently from the benefit to oneself (the "internal return"). This distinction allows a more precise estimation of the importance of altruistic concerns with others' payoffs. Some altruism is indicated by the fact that observed contributions to the public good increase as we raise the external benefit to others of one's own contribution, as reported in "Private Costs and Public Benefits: Unraveling the Effects of Altruism and Noisy Behavior" (Goeree, Holt, and Laury, Journal of Public Economics, 2002).  Contributing to a public good is analogous to investing in pollution abatement that will benefit others downwind or downstream.  Asymmetries in own (internal) and others' (external) benefits are evaluated in "Incentives in Public Goods Experiments: Implications for the Environment" (Goeree, Holt, and Laury, forthcoming 2003). 

Bargaining and Fairness
        Another context where non-selfish preferences are critical is in bilateral bargaining. In fact, Nash conducted his own bargaining experiments, and they largely failed due to fairness considerations. Our bargaining experiment is designed with asymmetric fixed payments that work in precisely the opposite direction from strategic considerations if participants are concerned with equity. The data from the experiments allowed us to estimate a logit equilibrium model in which players have a disutility from earning less than the other and a somewhat weaker disutility from earning more than the other. The parameters representing decision error, unfavorable inequality aversion ("envy"), and favorable inequality aversion ("guilt") were all statistically significant, and the resulting model tracked the observed pattern of proposals and responses. One implication is that models of decision error are complementary with the efforts of others to model preferences that are not perfectly selfish. This research is reported in "Asymmetric Inequality Aversion and Noisy Behavior in Alternating-Offer Bargaining Games," (Goeree and Holt, European Economic Review, 2000). 

Risk Aversion in Lottery Choice, Games, and Lottery Choice
        The recent working paper, "Quantal Response Equilibrium and Overbidding in Private-Values Auctions," (Goeree, Holt, and Palfrey, Journal of Economic Theory, 2002) applies theoretical advances to the study of a richer market-like setting, where bidders for a prize have randomly differing prize values. In an antique auction, for example, each bidder may know what they are willing to pay for a particular item, but the others' values depend on their own tastes and needs, which may not be observed. In the experiment, the throw of a die was used to determine whether the value of winning a prize was $0, $2, $4, ... etc, so each person saw their own value but not the others' values. The high bidder would earn the difference between their own prize value and their bid. The treatments vary asymmetries in the risks of raising or lowering bids in a manner that leaves the Nash equilibrium unchanged but alters the logit predictions in an intuitive manner: bids should increase when there is less "upside risk" from losing the auction by raising one's bid. In other words, asymmetries in payoff function "flatness" should have effects not predicted by standard theories of perfectly rational bidders who always choose the decision with the highest expected payoff, regardless of asymmetries in the risks of slight deviations. Think of it this way: the height of a hill represents expected profits, and a rational person is assumed to go to the highest point, even if it is on the edge of a sharp drop. 

        The most strident experimental controversy in the literature to date involved an exchange several years ago about risk aversion and flat payoff functions, and this paper revisits the issue by putting flatness on only one side of the payoff maximum. Consistent with economic intuition, bids are higher in the treatment with low upside risk, and lower in the treatment with high upside risk, despite the fact that the Nash equilibrium bids are the same for the two treatments. One striking feature of the logit analysis of auctions is that the predictions are so close to the aggregate bid distributions that we had to use thin lines in graphs to distinguish them. An interesting aspect of the data analysis is that there was overbidding by an amount that is rather precisely predicted by an assumption that bidders are risk averse. 

       The effects of risk aversion show up clearly in lottery-choice settings, where the subject chooses between pairs of lotteries with different degrees of riskiness.  This risk aversion is present in low-payoff settings (prizes of several dollars), and it increases sharply when all prizes are increased by a factor of 20, and it increases again when payoffs are scaled up by a factor of 50, yielding prizes in the hundred dollar range.  Moreover, risk aversion in high payoff situations is much stronger when the payoffs are "real" as opposed to being hypothetical.  These results are reported in "Risk Aversion and Incentive Effects in Lottery Choices" (C. Holt and S. Laury, American Economic Review, December 2002). 

        Many games involve situations where the potential payoffs from playing one strategy are much less variable than the potential payoffs from another strategy.  In "Risk Averse Behavior in Generalized Matching Pennies Games" (Goeree, Holt, and Palfrey, forthcoming in Games and Economic Behavior), we consider a game where all quantal response equilibria (for risk neutrality and any error rate) are on one side of the set of choice probabilities for the two players, but the data are clearly on the other side.  We prove that risk aversion shifts the stochastic best response functions toward higher probabilities of using the "safe" strategy.  The resulting quantal response equilibrium with risk aversion is close to the observed data average.  Maximum likelihood estimates of the coefficient of relative risk aversion are relatively stable across these different games. 
 

V. A Summary of Major Scientific Insights

        Game theory is the closest thing to a unifying theory in social science, and it evokes some of the strongest antagonism as well. Critics argue that people are not perfectly rational, and that the experimental support for game theory is mixed. The response of some theorists is that there must be something wrong with the experiments because the theory is logically correct. The problem with this normative defense is that what is optimal in a game such as the traveler's dilemma depends on what the other players actually do, not on what some theory says they should do. 

        Our research introduces randomness into decision making, which causes decisions to be only imperfectly related to measured economic incentives. Although the extent of this randomness is a matter of empirical estimation, the incorporation of such noise is a common element of the techniques that we use to describe the behavior of our laboratory subjects. This narrative has described three complementary modifications of classical game theory. The models of introspection, learning/evolution, and equilibrium contain the common stochastic elements that represent errors or unobserved preference shocks. These three approaches are like the "three friends" of classical Chinese gardening (pine, cherry, and bamboo), they fit together nicely, each with a different purpose. Models of iterated noisy introspection are used to explain beliefs and choices in games played only once, where surprises are to be expected, and beliefs are not likely to be consistent with choices. With repetition, beliefs and decisions can be revised via learning or evolution. Choice distributions will tend to stabilize when there are no more surprises in the aggregate, and the resulting steady state constitutes a noisy equilibrium. This general approach can usefully incorporate a broad range of human motivations, including altruism, fairness, and risk aversion, factors which improve the predictive power of equilibrium models in some of the applications we have considered. To summarize, the major scientific insights are: 

1) We have developed of a new model of noisy introspection to explain anomalous behavior in games played once. This introspective solution exists and reduces to the logit quantal response equilibrium in the special case where uncertainty does not increase with successive iterations of introspective thinking. 

2) We have used the logit equilibrium to explain anomalous data patterns in a range of games, especially those with a continuum of strategies, where we derive existence, uniqueness, and comparative statics proofs. The advantage of this approach is that the same formal model tracks behavior that closely conforms to Nash equilibrium predictions in one treatment and deviates sharply in another. 

3) We have developed ways to incorporate broader motivations into the stochastic equilibrium models, e.g. altruism, inequity aversion, and risk aversion, with applications to public goods, bargaining, and auctions. 

4) We have used learning models to explain the time patterns of directional adjustment in specific laboratory experiments. We have characterized a steady-state learning equilibrium and its relationship to the (static) logit quantal response equilibrium. 

5) For potential games, we use an entropy measure of dispersion to construct a "stochastic potential function" that is maximized by a noisy evolutionary process. All maxima of this stochastic potential are logit equilibria, which generalize the popular notion of "risk dominance" for selecting among multiple equilibria in 2x2 games. 

        These theoretical perspectives allow us to predict initial play, adjustment patterns, and final tendencies in a series of laboratory experiments. Data patterns that our colleagues would previously characterize as "behavioral" (i.e. consistent with intuition about human cognition but not with economic theory) are being picked up by these new stochastic game-theoretic models. There are discrepancies, but the overall pattern of results is surprisingly coherent, especially considering that we are using human subjects in interactive situations. The results are often precise in comparison with data from difficult physical science experiments, where impurities and extraneous forces may be hard to control. In fact, the principal investigator with a second degree in physics (Goeree) sometimes remarks that he is getting "that old physics feeling" when something unexpected happens in an economics experiment. 

        Laboratory experiments have been intimately connected with the development of game theory, starting with the reaction to Nash's seminal theorem. Two of the three recipients of the first Nobel Prize in Economics given to game theorists (Nash and Reinhard Selten) conducted experiments, and the third (John Harsanyi) was instrumental in the incorporation of random elements into equilibrium models. Patterns of human data provide the landmarks that are needed to avoid becoming lost in the jungle of possibilities once theorists move away from assumptions of perfect rationality. The resulting models have the empirical content that makes them relevant for playing games, not just for doing theory. 
 

VI. Abstracts of Recent Research Papers on Stochastic Game Theory

General Overview

Goeree, Jacob K., and Charles A. Holt (1999)  "Stochastic Game Theory: For Playing Games, Not Just for Doing Theory"Proceedings of the National Academy of Sciences, 96(September), 10564-10567, http://www.pnas.org/cgi/reprint/96/19/10564. Abstract: This paper argues that noisy models of introspection, learning, and equilibrium can explain the salient behavior patterns in game experiments, patterns that are not predicted by the Nash equilibrium or its refinements. Models of iterated noisy introspection are used to explain initial choices, models of noisy learning and evolution are used to predict dynamic adjustment paths, and logit equilibrium models explain Nash-invariant treatment effects in steady-state distributions of decisions. 

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Logit Equilibrium Models of Anomalous Behavior: What to Do when the Nash Equilibrium Says One Thing and the Data Say Something Else," in Handbook of Experimental Economics Results, edited by C.R. Plott and V.L. Smith, New York: Elsevier Press, forthcoming. Abstract: This paper shows how the logit equilibrium can be used to explain anomalous data patterns in a wide variety of games such as social dilemma games, coordination games, contests, pricing games, etc. 

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Bounded Rationality in Markets and Interactive Systems," Experimental Economics, forthcoming. Abstract: This is a selective survey of the literature on bounded rationality. Results from experimental economics are used to motivate models that relax the rational choice assumptions.
 

Noisy Introspection

Goeree, Jacob K., and Charles A. Holt (1999)  "A Model of Noisy Introspection," forthcoming, Games and Economic Behavior. Abstract: The paper presents a theoretical model of noisy iterated introspection designed to explain behavior in games played only once. The equilibrium determines layers of beliefs about others' beliefs about ..., etc., but relaxes the Nash-like requirement that belief distributions coincide with distributions of decisions, i.e. it allows for systematic surprises.  We prove that this thought process converges.  Data from 37 one-shot matrix games are used to estimate error and introspection parameters, which allows rejection of parameter restrictions implied by Nash and logit equilibrium limit cases.  Adding an introspection parameter cuts the mean squared prediction error by about half, and adding a risk aversion parameter reduces the remaining mean squared error by half. 

Goeree, Jacob K., and Charles A. Holt (2001)  "Ten Little Treasures of Game Theory, and Ten Intuitive Contradictions,"American Economic Review, 91(5), 1402-1422  Abstract: The "treasures" are ten static and dynamic games where behavior matches the Nash equilibrium or relevant refinement, and the contradictions show anomalous behavior patterns. In some games, Nash seems to only work by coincidence, e.g. if deviation losses are symmetric or very high, and in other games the data are repelled from the Nash prediction and pile up on the opposite side of the set of feasible decisions. 
 

Quantal Response Equilibrium

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (2002) "The Logit Equilibrium: A Perspective on Intuitive Behavioral Anomalies," Southern Economic Journal, 69(1) 21-47.Abstract: The paper characterizes logit equilibria for a class of N-player games with payoffs that depend on the ranking of player's decisions, including Bertrand price competition, Hotelling's location game, the traveler's dilemma, and many variants of coordination games. General existence, symmetry, uniqueness, and comparative statics proofs are presented and applied. 

Anderson, Simon P., Jacob Goeree, and Charles A. Holt (1998) "Control Costs and Equilibria in Games with Bounded Rationality," Discussion Paper, presented at the Summer 1998 ESA Meetings. Abstract: Van Damme's notion of control costs is that it is more costly to implement decisions more precisely. This paper derives the relationship between control costs and noisy approaches to equilibrium in games. In two-by-two games, quantal response equilibria are equivalent to Nash equilibria with control costs. Extensions to N-player matrix games are discussed. 

Goeree, Jacob K. and Charles A. Holt (2000)  "An Explanation of Anomalous Behavior in Binary-Choice Games: Entry, Voting, Public Goods, and the Volunteer's Dilemma," Discussion Paper. Abstract: This paper characterizes behavior with "noisy" decision making for a general class of N-person, binary-choice games. Applications include: participation games, voting, market entry, binary step-level public goods games, the volunteer's dilemma, etc. Many anomalous data patterns in laboratory experiments based on these games can be explained using the quantal response equilibrium. 

Goeree, Jacob K., Charles A. Holt, and Thomas Palfrey (2000)"Risk Averse Behavior in Generalized Matching Pennies Games," forthcoming in Games and Economic Behavior. Abstract:  We consider a 2 × 2 game in which each player chooses between a relatively safe decision and a risky decision with a large difference between the possible payoffs.  If players' utilities are correctly measured by financial rewards, the unique Nash equilibrium and all quantal response generalizations reside on one half of the set of choice probabilities.  The observed choice frequencies are well away on the other side.  Risk aversion shifts the stochastic best response functions toward each player's safe strategy, and the resulting intersection of these functions is close to the observed data. 
 

Learning and Evolution

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1997) "Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning," Discussion Paper. Abstract: This paper presents a dynamic model in which agents adjust their decisions in the direction of higher payoffs, subject to random error. This process produces a probability distribution of players' decisions whose evolution over time is determined by the Fokker-Planck equation. The dynamic process is stable for all potential games, a class of payoff structures that includes several widely studied games. In equilibrium, the distributions that determine expected payoffs correspond to the distributions that arise from the logit function applied to those expected payoffs. This "logit equilibrium" forms a stochastic generalization of the Nash equilibrium and provides a possible explanation of anomalous laboratory data. 

Capra, C. Monica, Jacob K. Goeree, Rosario Gomez, and Charles A. Holt (2002)  "Learning and Noisy Equilibrium Behavior in an Experimental Study of Imperfect Price Competition," International Economic Review, 43(3), August, 613-636 Abstract: The experiments implement a model of imperfect price competition in which the high price seller matches the lower price but has a lower sales quantity. The Nash-Bertrand prediction is unaffected by the market share of the high-price seller, but the data respond sharply to changes in this parameter, a response that is consistent with dynamic and equilibrium models of noisy behavior. Simulation techniques are used to explore conditions under which the steady-state "stochastic learning equilibrium" matches the logit equilibrium, and conditions under which there are small but systematic differences due to "recency effects." 

Capra, C. Monica, Jacob K. Goeree, Rosario Gomez, and Charles A. Holt (1999)  "Anomalous Behavior in a Traveler's Dilemma," American Economic Review, 89:3, June, 678-690. Abstract: The observed choices in a traveler's dilemma experiment are moved across the entire set of feasible decisions by changes in a treatment variable that has no effect on the unique Nash prediction. Dynamic patterns are explained by a logit learning model, and the steady state distributions are centered around the predictions of a logit equilibrium that generalizes the Nash equilibrium to allow for noisy behavior. 

Goeree, Jacob K. and Charles A. Holt (2000) "Stochastic Learning Equilibrium," Discussion Paper Presented at the June 2000 ESA Meeting .  Learning and adaptive behavior in an interactive context may be noisy, due to errors in forecasting and/or decision-making.  This paper introduces the notion of a stochastic learning equilibrium, which is a steady state distribution of players' histories of past decisions.  We prove existence and show that the steady states do not coincide with standard equilibrium predictions, e.g. Nash or quantal response, except under specific conditions.  The model is general enough to include endogenous learning rules and more mechanical reinforcement learning.
 

Rent Seeking

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "Rent Seeking with Bounded Rationality: An Analysis of the All Pay Auction,"Journal of Political Economy, 106:4, August, 828-853. Abstract: When people expend real resources to compete for a prize, a significant part of the associated value (rent) may be dissipated in the process. A model of bounded rationality is used to explain why the extent of rent dissipation in an all-pay auction may be sensitive to factors such as the number of competitors, the cost of effort, etc., which have no affect in a Nash analysis. A logit equilibrium model of boundedly rational behavior is proposed and analyzed. 

Goeree, Jacob K., Simon P. Anderson, and Charles A. Holt (1998) "The War of Attrition with Noisy Players," in Advances in Applied Microeconomics, Volume 7, edited by M.R. Baye, Greenwich, Conn.: JAI Press, 15-29. Abstract: We show that the Nash equilibria for the two-player normal-form war of attrition with asymmetric values involve one player choosing zero effort (conceding immediately). Non-degenerate mixed-strategy equilibria under different prize values are possible only when there is no maximum effort. These equilibria have perverse comparative static properties: an increase in one player's value leaves that player's bid distribution unaffected and raises the other player's effort. We describe the logit equilibrium for the game, which is symmetric when values are equal and predicts that the player with the higher prize value exerts more effort in the asymmetric case. 
 

Coordination

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (2001) "Minimum-Effort Coordination Games: Stochastic Potential and the Logit Equilibrium,"Games and Economic Behavior, 34(2), 177-199. Abstract: This paper considers coordination games with a continuum of Pareto-ranked Nash equilibria. The introduction of noise error yields a unique distribution of decisions that maximizes a stochastic potential function (expected value of the potential of the game plus entropy). As the noise vanishes, the limiting distribution converges to an outcome that is analogous to the risk-dominant outcome for 2×2 games. In accordance with experimental evidence and economic intuition, our results show that efforts decrease with increases in effort costs and the number of players, even though these parameters do not affect the Nash equilibria. 

Goeree, Jacob K., and Charles A. Holt (1999)  "An Experimental Study of Costly Coordination," Discussion Paper. Abstract: We present and test a unified view of behavior in coordination experiments with effort-cost and numbers effects not predicted by the Nash equilibrium. The theory is a generalization of risk dominance and maximum potential; the maximization of stochastic potential explains (final period) effort levels of human subjects in a series of minimum- and median-effort coordination experiments. 
 

Public Goods

Anderson, Simon P., Jacob K. Goeree, and Charles A. Holt (1998) "A Theoretical Analysis of Altruism and Decision Error in Public Goods Games,"Journal of Public Economics, 70:2 (November), 297-323. Abstract: This paper formalizes an equilibrium model in which altruism and decision-error parameters determine the distribution of contributions in public goods games. We prove existence of a unique, symmetric equilibrium density of contributions and show that (i) contributions increase with the marginal value of the public good, (ii) total contributions increase with the number of participants if there is altruism, and (iii) mean contributions lie between the Nash prediction and half the endowment. These predictions, which are not implied by a standard Nash analysis, are roughly consistent with laboratory data. 

Goeree, Jacob K., Charles A. Holt, and Susan K. Laury (2002) "Private Costs and Public Benefits: Unraveling the Effects of Altruism and Noisy Behavior," Journal of Public Economics, 83(2) 257-278 Abstract: The effects of a contribution to a public good are decomposed into an internal return to the contributor and an external return to each of the others. Contributions in one-shot games are generally increasing in internal returns, external returns, and group size, and a logit model of individual behavior tracks treatment averages well, both for linear and non-linear altruism specifications. 

Goeree, Jacob K., Charles A. Holt, and Susan K. Laury (2000)"Incentives in Public Goods Experiments: Implications for the Environment," forthcoming in Frontiers of Environmental Economics, J. List and A. de Zeeuw, eds..  Abstract:  This paper reports results of an experiment designed to evaluate the effects of externalities by altering the costs and benefits of an investment that corresponds to pollution abatement.  In this experiment, a person can make an investment with a private (internal) return that does not cover the investment cost, but with a public (external) return that makes the investment socially optimal.  Investments are increasing in internal and external returns, and are virtually identical for two treatments with the same "price," defined to be the ratio of the external benefit to the internal loss from making an investment. Individual forecast data make it clear that many who invest nothing are free riding on anticipated investments by others, and most who make significant investments in the final period do not expect others to be as generous. 
 

Bargaining and Auctions

Goeree, Jacob K., and Charles A. Holt (2000) "Asymmetric Inequality Aversion and Noisy Behavior in Alternating-Offer Bargaining Games," European Economic Review, 44, 2000, 1079-1089.  Abstract: We report a two-stage alternating-offer bargaining experiment in which players receive asymmetric fixed money payments in addition to their earnings from the bargaining process. These endowments do not affect the perfect positive correlation between initial Nash offers and the remaining pie, but are selected to induce a perfectly negative relationship between the remaining pie size and the first-stage offer that would equalize final earnings of the two players. This negative relationship is apparent in the data, which suggests the importance of fairness considerations. A model of asymmetric inequality aversion and stochastic choice is used to provide estimates of utility and logit error parameters. 

Goeree, Jacob K., Charles A. Holt, and Thomas R. Palfrey (2002) "Quantal Response Equilibrium and Overbidding in Private-Value Auctions." Journal of Economic Theory, 104(1), 247-272. Abstract: This paper reports the results of a private-values auction experiment in which expected costs of deviating from the Nash equilibrium bidding function are asymmetric, with the implication that upward deviations will be more likely in one treatment than in the other. Overbidding is observed in both treatments, but is higher in the treatment where the costs of overbidding are lower. We specify and estimate a noisy (logit) model of equilibrium behavior. Estimated noise and risk aversion parameters are consistent across treatments and are highly significant and of reasonable magnitudes. Alternative explanations of overbidding are also considered. The estimates of parameters from a cumulative probability weighting function yield a formulation that is essentially equivalent to risk aversion. 
 

Brief Survey Papers on Topics in Experimental Economics
(citations and coauthors are listed on my vita

     Methodology
        "Varying the Scale of Financial Incentives Under Real and Hypothetical Conditions"

     Industrial Organization:
         "Predatory Pricing: Rare Like a Unicorn?"

         "The Exercise of Market Power in Laboratory Experiments"

         "The Effects of Collusion in Laboratory Experiments"

         "Industrial Organization: A Survey of Laboratory Research"

          "Markets with Posted Prices: Recent Results from the Laboratory"

     Public Goods:
         "Theoretical Explanations of Treatment Effects
           in Voluntary Contributions Experiments"

          "Voluntary Provision of Public Goods: Experimental Results 
           with Interior Nash Equilibria"

     Games and Information:
         "Stochastic Game Theory: For Playing Games, Not Just for Doing Theory,"

         "Information Cascade Experiments"

          "Game Theory: Ten Little Treasures and Ten Intuitive Contradictions"

         "Logit Equilibrium Models of Anomalous Behavior: What to do when the 
           Nash Equilibrium Says One Thing and the Data Say Something Else"