**Summary**

We saw in section 2.5 that Gaus gives his refutations of attempts to derive principles of morality with revisionist accounts of rational choice in interactive contexts (again, game theoretic contexts). In section 2.6 Gaus considers what he takes to be the remaining option for the instrumentalist program. Here Gaus starts by claiming that if the instrumentalist employs the orthodox account of rational choice in games, then in order to try to explain moral norms as the product of instrumental rationality she needs to consider repeated interactions. In game theoretic parlance, the instrumentalist needs to employ the *folk theorem*.

The folk theorem, or folk theorems (some authors use the singular, other the plural), is actually a body of results in game theory. What these theorems tells us is that in a game repeated indefinitely often, if the players’ probabilities of continued play are sufficiently high (probabilities known as *discount factors*), then any outcome where their payoffs are all at least as great as their payoffs at a *minimax point* can be sustained in equilibrium. The minimax point is an outcome where all try to keep their partners’ payoffs as low as possible, and one can think of this as an outcome of “mutual punishment”. If, for example, the base game is the 2-player Prisoners’ Dilemma (and *C* is cooperate and *D* is defect) with appropriately bounded payoffs, then all outcomes where the players’ payoffs are at least as great as the payoffs of the outcome (*D*,*D*) can be part of an equilibrium of the indefinitely repeated game. A special and particularly famous case is the outcome where a pair of fixed partners in a repeated Prisoners’ Dilemma follow the “tit for tat” strategy. (A “tit for tat” player follows *C* on the first round of play and then always follows the choice the partner made at the previous round.) Given appropriate discount factors, if both follow “tit for tat” then they are at an equilibrium of the repeated game where each follows *C* at each round, although of course of course *C* is never part of an equilibrium of the one shot Prisoners’ Dilemma.

The folk theorem gets its name because game theorists discussed a number of these results informally before the first relevant theorems were published. As a historical tidbit, John Nash appears to have been the first to discover a folk theorem that shows that cooperation can be sustained in equilibrium in a repeated Prisoners’ Dilemma. Nash shared this result in an unpublished 1950 correspondence. Several authors, including Sugden (*Evolution of Rights, Cooperation and Welfare* 1986), Skyrms (‘The Shadow of the Future’ 1998) and Binmore (*Natural Justice* 2005) attribute proto-folk theorem reasoning to Hobbes in his response to the Foole in *Leviathan* 15 and to Hume in his analysis of promises in *Treatise* 3.2.5. (This may make it more apparent why I connected sections 2.5 and 2.6 to the so-called reconciliation project in the post on section 2.5.) The folk theorems look like exceptionally powerful results for the instrumentalist program, and in a certain sense they are. They tell us that the members of a community can follow an equilibrium in their repeated interactions where each must follow a cooperative strategy (like *C* in Prisoners’ Dilemma) with those who are in “good standing” and follow a punishing strategy (like *D* in Prisoners’ Dilemma) with those in “bad standing”, and this appears to vindicate the instrumentalist claim that norms of social morality really are the product of instrumental rationality. Indeed, some authors even claim that Hobbes’ system of natural law is at bottom a set of rules for following equilibria of the appropriate repeated games!

However, the folk theorems are only *existence* theorems. They tell us that in repeated games there are equilibria of mutual cooperation, but they tell us nothing regarding how we might start to follow these equilibria. Moreover, in a sense the folk theorems prove “too much”, because they also show that in repeated interactions there are a host of other equilibria besides equilibria of mutual cooperation, and at many of these equilibria some actually exploit others a significant amount of the time. Why should rational agents settle into “nice” equilibria of mutual cooperation rather than more “nasty” equilibria where some take advantage of others or even “state of nature” equilibria where no one cooperates with anyone else?

The instrumentalists who appeal to the folk theorem turn to evolution, and more specifically cultural evolution, as a means of explaining the appropriate equilibrium selection in the repeated games. As Gaus notes, Axelrod brought this idea to prominence in his pioneering study *Evolution of Cooperation* (1984). As Gaus also notes, many mistakenly concluded from Axelrod’s findings that “tit for tat” is the “correct” strategy for playing repeated Prisoners’ Dilemmas. In fact, no one strategy can be the uniquely rational strategy for playing repeated Prisoners’ Dilemma. There is not even a unique strategy for following a cooperative equilibrium of repeated Prisoners’ Dilemma. Gaus notes that the “grim” strategy also characterizes a cooperative equilibrium. In fact, there are many other strategies besides “tot for tat” and “grim” that characterize cooperative equilibria.

Gaus thinks that rules that require conditional cooperation are unlikely to evolve within human communities whose members are solely concerned with pursuing their own goals. Gaus will devote much of his discussion in Chapters III and IV to defending the claim that humans are just as fundamentally rule driven as they are purpose driven. If Gaus is right, then he has completed his overall argument against the instrumentalist program. But why does Gaus reject the proposal that instrumental rationality and cultural evolution are fundamental building blocks and the rules of social morality are entirely a product of these building blocks? As I read him, Gaus’ overall answer to this question is that rules that we would recognize as requirements social morality can emerge in communities of purely instrumentally rational agents only under “ideal” conditions, and in fact conditions are almost never ideal in the relevant sense. The “ideal” conditions Gaus alludes to in his discussion include perfect common knowledge, perfect homogeneity in group member preferences and possibly some combination of these. I’ll briefly consider preference homogeneity first, although Gaus discusses this near the end of this section. Gaus notes that a variety of authors have proposed that instrumentally rational agents in Prisoners’ Dilemma situations regard themselves (perhaps because of repeated play) as actually being in a Stag Hunt so that following the cooperative strategy is to follow one’s part of the optimal equilibrium. Gaus has reservations regarding this proposal. But as he notes, even if the “real” game for instrumentally rational agents to consider is the Stag Hunt, it is by no means a foregone conclusion that players will settle into the optimal equilibrium of mutual cooperation since at the equilibrium where none cooperate is the “risk avoiding” equilibrium of Stag Hunt. Beyond this, if the population is heterogeneous in the sense that some of the agents involved do not regard themselves as being in a Stag Hunt, this could lead to none of them cooperating in the end. Gaus discusses an unpublished essay by Kavka that discusses the *Assurance Dilemma* (“Political Contractarianism” 1989) together with a related study I worked on in support of this claim.

In this section Gaus also rightly stresses the importance of reputation and *indirect punishment*. In most realistic circumstances, we can to some extent choose our interaction partners. So if one crosses a partner in violation of a community norm, any consequent punishment may have to come from third parties, and this indirect punishment presupposes some knowledge of who is “in bad standing”. Gaus reviews some of the finding in Heinrich’s remarkable study of the Chaldean community of Detroit (*Why Humans Cooperate* 2007) and notes a number of serious inefficiencies in the Chaldeans’ practice of indirect punishment. For instance, many of the Chaldeans are able to “fake” cooperation by only nominally serving in tasks, and entire families are often punished for the perceived transgression of only one family member. So a significant fraction of innocent Chaldeans wrongly end up in “bad standing” and are consequently punished, while a significant fraction of the guilty remain in “good standing” and “freeload” off the Chaldean’s system of reciprocity. The Chaldeans have difficulty establishing accurate reputations because they rely on informal communication mechanisms such as gossip that do not always transmit information either widely or correctly. Gaus also discusses another related study I worked on where in a computer model of repeated community interaction, agents who exploit partners fare better than the agents who follow a norm of cooperation because the exploiters can introduce false information into the system of gossip by which these agents build their reputations. In these case studies, the efficient norms that would be norms of social morality cannot prevail because the community members lack the knowledge that would enable them to follow these norms.

(A couple of side notes: (*i*) One can read Gaus’ discussion here as a contribution to a recent and larger debate (which I think is taking place mainly in the social sciences) over the question of *strong reciprocity*. Gaus will introduce and discuss the idea of string reciprocity explicitly in Chapter III. For the moment it suffices to say that proponents of strong reciprocity reject the idea that moral norms can be explained entirely via a folk-theorem analysis. (*ii*) Most of the analytical and experimental work on repeated games focuses on the special case where players repeat the game always with the same partners. Kandori published a landmark study (‘Social norms and community enforcement’ 1992) where he proved the central relevant folk theorems for repeated games where community members interact repeatedly with partners that might change over time, allowing for the enforcement of social norms via indirect punishment. Kandori explicitly acknowledges that the cooperative equilibria of community enforcement presuppose that all in the community can accurately distinguish between those in “good standing” and those in “bad standing”.)

**Questions**

Is the case for the folk theorem analysis better if the agents can generate common knowledge? In my opinion, the failures of the evolution of norms of social morality that Gaus discusses stem from the agents lacking the appropriate common knowledge. For instance, in cases like the heterogeneous population models of my own that Gaus discusses where some *moderate* individuals have Stag Hunt payoffs and other *dominating* individuals have Prisoners’ Dilemma payoffs, all fail in the end to cooperate with each other because they cannot distinguish between the dominators that are actively trying to exploit others and the moderates who fail to cooperate only in self-defense. Consequently they are unable to follow contingency strategies where they cooperate with those in “good standing”. But what if such individuals *can* reliably distinguish between those in good standing and those in bad standing? Gaus observes that the wealth of analytical and experimental studies on the evolution of cooperation supports the claim that effective cooperative norms can emerge in smaller communities with effective communication networks, but such norms are not likely to emerge in larger communities that lack effective such communication networks. But suppose such communication networks emerge in larger communities, so that certain facts (like who in the community is “guilty”) can be made common knowledge? The analysis of network evolution is admittedly in its infancy, and I think even less work has been done on the analysis of communication networks than on interaction networks. On the other hand, there are also important empirical studies that show that if the relevant structures for knowledge transmission are in place, then a folk theorem analysis can explain how norms of cooperation are sustained even among communities of agents who belong to widely dispersed cultures and meet for trade only occasionally. Might perhaps future research lend more support to the folk theorem analysis?

Does Gaus’ larger project require him to reject the folk theorem analysis? As I have hinted, in my own opinion the jury is very much still out on the folk theorem analysis of social norms. Gaus has presented some particularly important criticisms of this analysis, and there are others in the literature. And, not surprisingly, some of the critics of the strong reciprocity idea defend the folk theorem analysis against many of these criticisms (though not Gaus’ specific criticisms yet, since his work is brand new). But I think perhaps a more interesting question to ask is to what if any extent Gaus’ overall project depends upon his rejecting this part of the instrumentalist program. The folk theorem shows that a huge number of different equilibria of repeated interactions are possible. The challenge for a supporter of using folk theorems to explain social morality is to give an appropriate account of *equilibrium selection*. Gaus argues that the evolutionary accounts of the sort he discusses in this section will not succeed, and I’ve suggested that this conclusion is premature. But Gaus has his own, quite different, account of how norms of social morality can evolve in Chapter VII. Just how is Gaus’ evolutionary account different from those he criticizes in this section? And does Gaus need to jettison the folk theorem analysis in order to introduce his own evolutionary analysis of social morality? (I have my own ideas regarding these questions but I think I should let others weigh in here.)

Game theorists have for some time now been making use of reputation in order to derive certain notions. What is reputation? GG does not tell us very much (nor do most game theorists). He talks about reputations for trustworthiness and the like (pp. 90, 95). How are we to understand that? In every day life, when I tell you that Peter V is trustworthy or honest, I say something about his character. (“He would not do that!”) But there is no place for character in a world where agents are characterized by their utility functions and their assignment of subjective probabilities.

One could understand “reputation” as encoding information about how people

behavein a variety of circumstances. This notion rational choice theorists are entitled to. But that’s not what we ordinarily mean when we talk about, say, trustworthiness.I look forward to chs. 3 and 4. I expect GG will surprise us.

As I’m sure many of you know already, Peter Vanderschraaf has done some really sophisticated and interesting work on reputation and reciprocity, see especially his “The Invisible Foole” in Phil Studies. One of the important questions when dealing with reputation, as Peter indicates, is the strength of the common knowledge of reputations and the propensity of agents to make errors in assessing reputations, especially in larger groups. I would be curious to her how Peter thinks that Gaus’ approach is either in tension with or completed by Gintis’ analysis of common knowledge and the folk theorem in

Bounds of Reason.