Estlund Reading Group Chapter 12

If your account of democratic authority uses the term “epistemic” then sooner or later you’re going to have to deal with the Jury Theorem. And here is where David takes up the gauntlet.

I’ve made that seem rather dramatic, but by this point in the book the gauntlet isn’t especially heavy! After all, in preceding chapters we’ve seen a model form of deliberation, a distinction between formal and substantive epistemic value, and a careful distinction between a “test” for finding the correct answer to some shared problem (such as majority rule), and a “testing system” (such as a constitutional democracy within which majoritarian decision procedures are embedded). The Jury Theorem, as tantalizing as it may be to some democratic theorists, does not appeal to discussion and argument, relies on claims about voter competence and the substantive correctness of some choices, and it applies in the first instance to specific tests (voting and majority or plurality rule), not to a testing system per se.

Now to be sure, if the theorem gave us great confidence in the test (or a series of related tests: majority rule, plurality, Borda count, etc), then we might want to tailor our testing system to give the relevant testing mechanisms pride of place. But David does a good job of showing why such confidence may, in the final analysis, be misplaced: we have insufficient grounds for supposing that voter competence is, as a general rule, reliably greater than random; and insofar as we can sustain such an assumption, we end up appealing to mechanisms that do the epistemic work for us, thus rendering the theorem itself uninteresting.

I find David’s critique of the Jury Theorem on the whole convincing, although I think part of the argument can be presented in a way that makes the connection to the rest of the book more intuitive, and I try to make this connection. I conclude with a sceptical thought about where we now stand in the argument for epistemic proceduralism, framed in light of the concluding remarks in this chapter. (Much of what is said in this chapter against the Jury Theorem is also cast against a Bayesian approach; that discussion is very brief and seems plausible, so I’ll focus on the Jury Theorem.)

Suppose we are voting on a question that can reasonably be thought to have a correct and an incorrect answer, and we are choosing between two options (which of these two candidates is the best leader? is this defendant innocent or guilty?). Suppose further that a great many voters prefer the correct answer and vote sincerely in pursuit of that answer, as they understand it. If each voter is at least marginally better at choosing the correct answer than a coin flip, and if their choices are (statistically, not necessarily causally) independent, then the likelihood of the majority choosing the correct answer approaches one as the number of voters increases.

Indeed, we don’t even need to have every voter possessing a competence of p > 1/2, so long as the distribution of competence is roughly (gaussian) normal and the average competence does exceed p = 1/2. Even these distributional constraints can be loosened, and it turns out we can also weaken the assumptions of independence, common preference, and sincere voting, and still get promising results, as a series of recent papers has shown (of which Roger Myerson’s 1998 paper on “Extended Poisson Games and the Condorcet Jury Theorem” in Games and Economic Behavior stands out, in my judgement).

So the theorem is interesting, and versions of the result withstand considerable tinkering with the initial assumptions.

David succinctly explains the essence of the theorem and correctly links it to the law of large numbers (first demonstrated in its weak form by Bernoulli early in the 18th century: roughly speaking, this is the tendency of the sample mean of a random variable to converge to its expected value as the number of trials increases). He then traces a course between pessimists and optimists in detailing a key assumption required for this result to hold: the statistical independence of voter choices. This turns out to be a more complicated condition than either pessimists or optimists admit.

Pessimists tend to conflate causal and statistical independence, concluding that, because there are obvious instances of systematic influence in real-world political discussion and debate, statistical independence is highly unlikely. Optimists, in contrast, think it sufficient for independence that, at the end of the day, people make up their own minds rather than merely deferring to some authority. But we can imagine voters making up their own minds in this sense, yet routinely voting the same way along some issue dimension, thus violating statistical independence. Voters may also defer to some opinion leaders, yet do so with sufficient independence of judgement that, on the whole, statistical independence is satisfied. Or there might be situations where overall competence is improved to the extent that there is truth-tracking deference by some voters, who are competent at recognizing reliable opinion leaders; indeed, we might want more of that sort of deference, even if it violates statistical independence! Thus failures of independence don’t obviously lead us to reject the Jury Theorem as promising for democracy, but nor does the mere satisfaction of that condition necessarily sustain such promise.

What about binary choice? The jury theorem is most often framed as a property of binary choices (yes or no, guilty or innocent), whereas many political choices are not binary, and even those presented as binary are often contrived aggregations of complex questions, or the result of complex sequences of judgements, some binary and others not (think of the party primary system for nominating US presidential candidates). When making assumptions about voter competence in such settings, should we consider the binary choice? the expanded set of options implicit in that choice? or the entire range of judgements leading up to the binary choice at hand?

The difficulty of answering such questions without making strong background claims will turn out to be critical to David’s dismissal of the theorem, but he works his way up to that point in a balanced and thoughtful way, acknowledging the very interesting ways in which, as with independence, the story for non-binary choices is more complicated that it might at first appear.

Consider that, contrary to initial appearances, an intuitive extension of simple majority rule — plurality voting — is, in the words of Christian List and Robert Goodin (whose 2001 paper in the Journal of Political Philosophy looms large behind this chapter), an “epistemically eligible” choice rule. Why? Because under a plurality rule the winning option does not require an absolute majority, as it would in the binary choice setting. The law of large numbers applies as surely to a binomial random variable (two possible states with probabilities p and 1-p) as to a multinomial random variable (k possible states).

If, for instance, the average voter competence over three options is Pcorrect= 0.4 versus Pwrong1 = Pwrong2 = 0.3, then the same dynamic that generated a decisively correct majority decision in the two-option case will also work in the case of several options. As the number of voters increases, and if voter competence is at least slightly greater than random at choosing the correct option, then the correct option will garner more votes than either of the incorrect options.

To be sure, the truth-tracking tendency of plurality voting over several options is not nearly as striking as in the binary choice setting, where the likelihood of the majority being correct approaches one rather dramatically as the number of voters increases. Yet modern electorates are big. Very big. Even municipal elections in many contemporary towns (and virtually all cities) feature populations of sufficient size to generate favourable epistemic outcomes, and not just by plurality rule: if List and Goodin’s simulations in their 2001 article are suggestive of the likely dynamics at work, then the Borda count, or pairwise contests (Condorcet’s favoured procedure, clarified and elaborated by H. Peyton Young) likely also do pretty well at tracking truth.

David acknowledges these very interesting and suggestive results: the source of his scepticism lies elsewhere. There are good reasons to doubt that voters are, as a general rule, better than random at choosing correctly. We know there are a variety of ways that voters can be systematically mistaken, from factual errors to systematic ideological influences that may favour correct answers on some issues, but fail dramatically on others. And even if we are optimistic about, say, the successful use of informational heuristics to solve these sorts of problems, there is a deeper worry: for sufficiently complex problems, it isn’t at all obvious how we should define “better than random” competence, and many political problems are complex in just this way. This is “the disjunction problem”.

Suppose we have voters facing a choice over k options, one of which is correct. Suppose further that at least one of those options can be disaggregated into two or more further options. Now suppose we assume random average voter competence of 1/k in the initial choice situation. It seems to follow that we are in fact assuming greater than random competence for the choice situation were it disaggregated into disjunctive propositions involving, say, k+n options, for which random competence would be 1/(k+n). If the strength of our competence assumption can change so dramatically according to how we frame the choice situation, then shouldn’t we be worried?

On the face of it we might not see this as a problem at all, and in fact I’m not sure the discussion on pp. 229-230 makes the case for a problem here as convincingly as one could. After all, we might reply that the assumption of slightly better than random voter competence presupposes correct specification of the choice problem. A little thought suggests, however, that this might not be a very promising line of response. Are we really to suppose that, for all or most complex problems that come to a vote, there is a way to partition the full set of related options into subsets of options, such that among each subset there is a correct answer (to be discovered either by simple majority rule if the choice is binary, or plurality or some other decision rule, if there are more than two options in a given subset)? And even if we could reliably partition all or most complex moral, political, and economic issues in this way, is it really reasonable to suppose that average voter competence at each stage is at least slightly better than random?

If we believe it is so possible, and that the competence assumption over properly delimited subsets is reasonable, then we might recommend tailoring the political system in such a way that all decisions put to the electorate are presented as correctly delimited subsets of complex problems, so that each decision fits together to yield the best solution to the complicated matter at hand. And it isn’t crazy to think that scholars might go down this path: consider some of the enthusiasm around deliberative polls, where the problem is to structure the choice situation, informational environment, and participant deliberations in ways that better get at what informed and civil citizens would think on an issue. So this species of deliberative democrat arguably believes that there is, generally speaking, a truth of the matter that democratic procedures ought to track: the informed and reasoned voice of a representative body of citizens. And she further thinks it reasonable to assume better than random competence for voters when they are put in the correct informational environment and face properly framed questions.

Note, however, that taking this tack likely means addressing David’s rejection of an epistocracy of the educated — for who else but highly educated overseers and experienced technocrats are going to be called upon to correctly delimit complex social problems, so that they are fit to be voted upon under the assumption of better than random competence?

So I think the force of the disjunction problem isn’t as clear as it could be when we first encounter it. Subsequent remarks toward the end of the chapter seem to clarify what the disjunction problem is meant to illustrate, but one of those remarks is a bit confusing: “there is no privileged way to count alternatives that would warrant the intuitive thought that individuals must be at least better than random.” I see what is meant, but this formulation doesn’t seem quite right: there may well be privileged ways to do this, but the point of the disjunction problem, I take it, is that arguing for such privilege comes at a cost (“no competence assumption is available, so to speak, for free,” p. 233). I hope my take on the problem here makes clear how some of those costs would likely play out, and how the rest of David’s approach buttresses his scepticism about the Jury Theorem: if you think the disjunction problem is a non-problem, then you’ll end up having to argue for “some privileged way to count alternatives” (p. 231), and that will likely involve justifying epistemic privilege in ways that run up against the idea of qualified acceptance, and the argument against an epistocracy of the educated.

The real critical strength of this chapter, I believe, is the indictment that the Jury Theorem only aggregates information in the most spartan sense: the theorem in no way captures “the sharing of diverse perspectives” and “bringing together different parts of the puzzle” (p. 232), in the ways that we usually think of when singing the potential praises of inclusive deliberation and a public sphere characterized by diverse arguments and Mill-inspired `experiments in living’.

Bayesian approaches do feature a sort of communication, in the sense that voters use information about other voters’ choices to update their estimates about which option is correct. But that too is a clumsy way to capture what actually goes on in, say, scientific debates or public arguments, and simple bayesian updating can easily lead to perverse cascades that lurch voters toward falsehood as easily as toward truth. And although David doesn’t mention any of the extant work in formal political economy on the jury theorem, much of that literature is rife with talk of strategic voting and voters receiving “signals” about “states of the world.” Yet these approaches seem to me to be similarly barren: their formal renderings of signaling and updating processes simply fail to capture the rich dynamics of learning and persuasion (perhaps because many of them at heart assume that such dynamics amount to little more than cheap talk?).

By way of conclusion, David seeks to show that his favoured approach, epistemic proceduralism, is not waylaid by the very problems he has exposed for the Jury Theorem. Because epistemic proceduralism appeals directly to group competence, it need make no controversial assumptions about individual competence, as the Jury Theorem must. But does the disjunctive problem apply directly to group competence? David thinks epistemic proceduralism may escape some of the force of this potential application by identifying plausible mechanisms whereby group competence is reliably better than random: “interpersonal communication and reasoning about the question at hand” (233). And of course these were the mechanisms that are largely absent from the Jury Theorem, which relies only on an assumption about individual competence, paired with a statistical property of large distributions .

To be sure, “the epistemic value of communication” is not itself obvious: there are good reasons to think that some forms of communication could “undo the epistemic value of thinking together.” The point is “not to dismiss or deny the flaws, blind spots, and pathologies that are well known in studies of deliberation and communication in political and nonpolitical contexts” (179), but rather “to illustrate how an account of this kind could provide a basis for the epistemic value of group deliberation” (234).

So, to the advocate of better than random competence we pointed out that their assumption demands an argument for a “privileged way to count alternatives” faced by voters. That argument in turn must pass the formidable hurdles presented in earlier chapters: qualified acceptance, a model deliberative situation tailored to finding independently correct answers, and the rejection of epistocracy. To justify epistemic proceduralism against the disjunction problem, we thus frame the deliberative mechanisms that likely favour better than random group competence in light of the model deliberation framed in chapter nine.

But David isn’t yet satisfied with such a defense of his favoured view, because the attention in chapter nine was devoted to avoiding certain “primary bads”. The model of deliberation on offer there, paired with the discussion in chapter ten, together seem to give some faith that (sufficiently epistemic) real-world deliberation might support favourable group competence in avoiding primary bads, but we also need to know whether competence in avoiding bads is likely to translate into sufficient competence on other important public matters (building bridges, setting correct tax rates, and the like), or at least that the strength in avoiding bad things is enough to outweigh incompetence with respect to some less vital but still important public concerns.

We are, then, “left needing to maintain that group performance on important issues (avoiding primary bads) is, under favorable but possible conditions that epistemic proceduralism needs, and after public deliberation of certain kinds, enough better than random to outweigh, in the weighted score, any especially poor performance by the group in other areas” (235).

We do indeed need to maintain this to get the full bill of goods from David’s account, but if chapters six, nine, ten and eleven didn’t persuade you, then this will seem to be a rather modest conclusion! That is, if you remain sceptical of the more ambitious claims for epistemic proceduralism, then you will likely conclude that, while we still have no really knock-down case for inferring sufficient group competence beyond the avoidance of primary bads, the case at least isn’t wildly implausible, and regardless the Jury Theorem provides no comfort.

I’m not sure yet just how sceptical I am, and I admit to finding this a very attractive approach to democratic authority. That said, let me press a sceptical point. Consider the claim above that we might find group competence in avoiding bads sufficiently impressive that it outweighs incompetence in other areas. Presumably there will be qualified disagreements about the relative weighting of various issues and problems, even if we all agree on the primary bads. While we might find widespread agreement on the relative merits of democracy in avoiding really bad stuff, it seems far more controversial to argue for either epistemic spillover to, or priority over, other less-vital but still important public issues.

I’m going to indulge myself by revisiting my favourite reasonable minority (mentioned in my earlier contribution to our discussion): pansexual polyamorous pacifist vegans. Here is a group that I think could pass the hurdle of qualification for most of their views, yet their judgements on a many public issues will differ markedly from the mainstream, even in a reasonably diverse society. They reject the profound socioeconomic injustices implicated in meat-intensive diets and a militaristic political culture. They find monogamous heteronormative marital norms oppressive. They are hippies. Hardcore hippies. Overeducated, freeloving hippies. But they aren’t militant, or brazenly dismissive of other viewpoints and practices, nor are they unresponsive to civil and informed argument. They just happen to think that most such arguments lead obviously to moral and policy conclusions that the majority routinely rejects.

Members of this group will likely agree with the majority that democratic procedures tend to track the correct answers when it comes to avoiding primary bads, yet I suspect they would quite reasonably dispute the claim that such downside insurance easily vindicates the persistent errors that majorities make when deciding on taxes and bylaws and trade policies and the like. Group members might nonetheless accept democratic procedures as legitimate, in spite of the persistent errors they make in all but the most essential (primary bad avoiding) cases, but the justificatory burdens faced by this reasonable minority seem strikingly onerous compared to a complacent militaristic monogamist carnivore, happily nestled in a firm democratic majority on a great many issues over the course of his life. The majority can count on democratic procedures not only to tend away from primary bads with sufficient reliability, but also to track certain goods; for my freeloving peacenik vegans, only the first expectation is reasonable, and the second seems outright mistaken.

I’m not suggesting that our account of democratic authority and legitimacy has to be sufficiently responsive to every conceivable shade of reasonable opinion on every possible issue dimension (although why not?). And I think by this late chapter the case for the epistemic value of democratic procedures in avoiding primary bads is at the very least plausible. But the more ambitious claim that David asks us to maintain — that group competence on critical issues either spills over into other areas, or outweighs incompetence elsewhere — isn’t yet obvious; and even if it is at least plausible, we need to reconcile it with the sort of diversity that is easily consistent with the rather capacious acceptability requirement so essential to epistemic proceduralism.

Be Sociable, Share!
This entry was posted in Posts, Reading Group. Bookmark the permalink.

3 Responses to Estlund Reading Group Chapter 12

  1. Ben Saunders says:

    Thanks Loren for that truly great discussion. I have to say, I found pretty much everything David said in this chapter convincing, which unfortunately may not make for great discussion.

    One problem I’ve always had with the CJT is that it seems to assume not only binary options but one correct answer. The example you gave above Loren (from Goodin and List) was Pcorrect= 0.4 versus Pwrong1 = Pwrong2 = 0.3. My worry is about cases like Pcorrect1 = Pcorrect2 = 0.3 versus Pwrong = 0.4. Here it seems that, even though individuals have a greater than 50% chance of getting a right answer, if they’re split between the two equally right answers then the wrong outcome may win a plurality. Of course, this can be avoided by different electoral rules, e.g. STV, and in any case we’ve already seen plenty of reasons to reject CJT, but that’s just one that’s always bothered me.

    As for your own hobby-horse, these hippie vegans, it’s true that they’ll lose out on matters decided collectively through voting (or, possibly, the market). I guess what this really raises, however, is questions about liberalism, rather than democracy, insofar as the two are distinct. While they’re often portrayed as potentially conflicting (e.g. Berlin, Sen), democracy has been praised as allowing individual liberty since at least Pericles’ funeral oration. It may well be, therefore, that these people will be left to live their lives as they choose.

    Of course, as you point out (and this is the real crux of your argument, I take it), that doesn’t follow if we focus only on primary bads. If we simply pick the government that does the best job of avoiding war and famine, there’s no guarantee it won’t trample on civil liberties. I suppose one thing we could do is add major infringements of such to our list of primary bads (I can’t remember the list exactly, but I think it wasn’t intended to be exhaustive). Beyond that, however, we may simply have to say the hippies should be more concerned about avoiding war and famine than having the marriage laws they want*, and willing to recognize that, since they must live with a majority not sharing their lifestyle, they have no qualified objection to the arrangements David describes.

    *One problem with this being that primary bads are I take it defined in terms of being universally recognized as bad, which need not be the same as them being the most important bads. Lack of food is a primary bad, whereas lack of place-of-worship-for-religion-X isn’t.

  2. Loren King says:

    Ben: “One problem I’ve always had with the CJT is that it seems to assume not only binary options but one correct answer.”

    If I were a choice theorist (I’m not), I’d probably model this as nonuniform fundamental preferences. An easy CJT case has a unanimously desired outcome, but uncertainty about the best of two available means to that end. Yet there might well be several plausible ways to arrive at several available outcomes, and disagreement not only over means, but also about which outcome is indeed best. Pcorrect1 = Pcorrect2 = 0.3 and Pwrong = 0.4 would then be equivalent to disagreement about which of two correct means we should pursue. For US Democrats I suppose this is Clinton vs Obama (vs McCain. For Republicans, Pcorrect = 0.4, although perhaps only grudgingly, with some holding their noses as they vote, as his mother predicts). As you say, plurality voting runs into problems here, and while there are ways around this, they come with costs (domain restrictions, etc). I agree that this doesn’t get at the deeper concerns about the Jury Theorem.

    Vaguely relevant: in the American version of “Who Wants to be a Millionaire?” the audience, when polled by a stumped contestant, tended to do a lot better than I’d have thought likely with plurality voting in a CJT setting. My sense is that the questions typically had a correct answer, a plausible answer, and two silly options. Thus for most participants much of the time, these were essentially binary choices. But I remember a dramatic failure of group competence on a question about the capital of Australia: the options were Syndey, Melbourne, Canberra, and maybe Adelaide or Geelong? (I don’t remember). A strong plurality chose Sydney, followed by a solid chunk of votes for Melbourne, with a smattering of votes for the correct answer, and a few for the fourth option. Optimistic competence assumptions indeed come at a price.

    On the hippies, I guess I’m not worried that the focus on primary bads will allow threats to civil liberties per se, but I think you’re correct that my closing comments try to find a tension between liberal commitments and democratic authority, and then to invite speculation about whether or not epistemic proceduralism gives us sufficient resources to speak to that tension.

    My example suggests that there can be deep and enduring, yet qualified, disagreement over the relative epistemic merits of democratic procedures beyond the (obviously vital) downside insurance of avoiding primary bads, and I further wonder if this will have implications for stability. I think an account of democratic authority should have something to say about the question of stability (although that’s a problem somewhere between philosophical and social-theoretic concerns, perhaps drifting closer to the latter). Given the sort of reasonable diversity that an inclusive qualification standard permits, I wonder if the modest epistemic virtue of democratic procedures accounts for why persistent reasonable minorities accept (or ought to accept) the legitimacy of those procedures across a wide range of issues and concerns, over the course of entire lives and several generations?

    Here is a more pointed version of my worry that anticipates some of David’s final chapter: the experience of democracy seems to have turned my hypothetical hippies into hopeless realists! (pp. 264-267)

    The hippies, after all, hold both their institutions and their fellow citizens to standards several of which could actually be met (so they aren’t merely utopian); yet after decades, even generations of persistent argument followed by fruitless voting, they have good reason to believe that their (morally correct and actually attainable) standards will never, ever be met by these institutions (the legitimacy of which they nonetheless are expected to affirm).

    Many of the hippies may have started out in young adulthood viewing their favoured principles and policies in a hopeful way — as “appropriate standards which are not only possible for people and institutions to meet, but which there is no strong reason to think they will not meet” (267). The experience of putatively legitimate democratic institutions has (or should have) convinced them otherwise.

    Now that might actually be a good thing, if the hopeless view is morally better than a more hopeful alternative. David will make a case in the final chapter that we shouldn’t have a kneejerk preference for nonhopeless theories in philosophy; so why in our moral and political perspectives? But I cannot help but wonder if this might have some potentially perverse implications for longterm stability.

  3. I’ve just posted some remarks on Loren’s comments.

Leave a Reply