The Law of Large Numbers and infinite run payoffs

In discussions of maximization of expected value, the Law of Large Numbers is sometimes invoked, at times—especially by me—off-handedly. According to the Strong Law of Large Numbers (SLLN), if you have an infinite sequence of independent random variables X₁, X₂, ... satisfying some conditions (e.g., in the Kolmogorov version ∑_n(σ_n²/n²) < ∞, where σ_n² is the variance of X_n), then with probability one, the average of the random variables converges to the average of the mathematical expectations of the random variables. The thought is that in that case, if the expectation of each X_n is positive, it is rationally required to accept the bet represented by X_n.

In a recent post, showed how in some cases where the Strong Law of Large Numbers is not met, in an infinite run it can be disastrous to bet in each case according to expected value.

Here I want to make a minor observation. The fact that the SLLN applies to some sequence of independent random variables is itself not sufficient to make it rational to bet in each case according to the expectations in an infinite run. Let X_n be 2ⁿ/n with probability 1/2ⁿ and  − 1/(2n) with probability 1 − 1/2ⁿ. Then

• EX_n = (1/2ⁿ)(2ⁿ/n) − 1/(2n)(1−1/2ⁿ) = (1/n)(1−(1/2)(1−1/2ⁿ)).

Clearly EX_n > 0. So in individual decisions based on expected value, each X_n will be a required bet.

Now, just as in my previous post, almost surely (i.e., with probability one) only finitely many of the bets X_n will have the positive payoff. Thus, with a finite number of exceptions, our sequence of payoffs will be the sequence  − 1/2,  − 1/4,  − 1/6,  − 1/8, .... Therefore, almost surely, the average of the first n payoffs converges to zero. Moreover, the average of the first n mathematical expectations converges to zero. Hence the variables X₁, X₂, ... satisfy the Strong Law of Large Numbers. But what is the infinite run payoff of accepting all the bets? Well, given that almost surely there are only a finite number of n such that the payoff of bet n is not of the form  − 1/(2n), it follows that almost surely the infinite run payoff differs by a finite amount from  − 1/2 − 1/4 − 1/6 − 1/8 =  − ∞. Thus the infinite run payoff is negative infinity, a disaster.

Hence even when the SLLN applies, we can have cases where almost surely there are only finitely many positive payments, infinitely many negative ones, and the negative ones add up to  − ∞.

In the above example, while the variables satisfy the SLLN, they do not satisfy the conditions for the Kolmogorov version of the SLLN: the variances grows exponentially. It is somewhat interesting to ask if the variance condition in the Kolmogorov Law is enough to prevent this pathology. It’s not. Generalize my example by supposing that a₁, a₂, ... is a sequence of numbers strictly between 0 and 1 with finite sum. Let X_n be 1/(na_n) with probability a_n and  − 1/(2n) with probability 1 − a_n. As before, the expected value is positive, and by Borel-Cantelli (given that the sum of the a_n is finite) almost surely the payoffs are  − 1/(2n) with finitely many exceptions, and hence the there is a finite positive payoff and an infinite negative one in the infinite run.

But the variance σ_n² is less than a_n/(na_n)² + 1 = (1/(n²a_n)) + 1. If we let a_n = 1/n² (the sum of these is finite), then each variance is at most 2, and so the conditions of the Kolmogorov version of the SLLN are satisfied.

In an earlier post, I suggested that perhaps the Central Limit Theorem (CLT) rather than the Law of Large Numbers is what one should use to justify betting according to expected utilities. If the variables X₁, X₂, ... satisfy the conditions of the CLT, and have non-negative expectations, then P(X₁+...+X_n≥0) will eventually exceed any number less than 1/2. In particular, we won’t have the kind of disastrous situation where the overall payoffs almost surely go negative, and so no example like my above one can satisfy the conditions of the CLT.

The replay argument against agential control

Van Inwagen’s replay experiment is supposed to show that indeterministic free will is problematic. We imagine someone choosing betweeen A and B, then we rewind the state of the universe to just how it was before their choice, and repeat. Van Inwagen thinks that as we continue the experiments, we will get the proportion of choices that are As converging to some number, say 30%, and that tends to make us think that the choices are random rather than something in the agent’s control.

Let’s imagine we’ve actually done this replay a very large number of times. Here are the three possibilities for what we might observe:

• O₁: The proportion of As settled down to some number strictly between 0 and 1.

• O₂: All the choices were As or all the choices were Bs.

• O₃: The proportion of As changed and did not settle down.

For familiar libertarian reasons, it seems that O₂ would be evidence against the hypothesis that the choice is in the agent’s control: they both support the hypothesis that the agent is compelled or nearly compelled, either by their character or by external forces, to choose as they did.

Van Inwagen claims that observation O₁ is evidence against the hypothesis that the choice is in the agent’s control. For Bayesian reasons, given that O₁, O₂ and O₃ are exhaustive and mutually exclusive, and that O₂ is also evidence against the agent’s control, the only way this could be is O₃ favored the agential control hypothesis (unless our prior probability of O₃ is equal to 1, which it’s not).

But I don’t think observation O₃ would favor the agential control hypothesis. There are two reasons for my judgment.

First, let’s subdivide observation O₃ into two suboptions.

• O_3a: There is some discernible pattern to the As and Bs that precludes the proportion of As from settling down, e.g., ABBAAAABBBBBBBBAAAAAAAAAAAAAAAA...,

• O_3b: It’s just a complete mess and the proportion doesn’t settle.

Now, O_3a is a strange option. A pattern initially may seem like evidence of control. But once we recall that the experiment involved a rewinding of the universe’s state, we can see that any pattern that is not just all As or all Bs cannot be the result of agential control, since control of a pattern across time would require memory, and we have assumed that any memories are wiped. So, given the rewinding, we know that any pattern—other than the patterns that do not require memory, namely the patterns in options 2 and 3, is a fluke. Thus, 4a shouldn’t be evidentially different from 4b.

What about O_3b? Well, O_3b is more of a mess than O₁. A mess is, if anything, more random than a sequence with a probabilistic structure, and hence if anything less indicative of agential control. Thus, O_3b is either neutral with regard to the hypothesis of agential control or even evidence against it. And O_3b is in the boat by the above remarks. Thus, O₃ is either neutral with regard to agential control or evidence against it.

Second, the reason van Inwagen thinks O₁ goes against the agential control hypothesis is because it favors the hypothesis that we simply have a probabilistic structure of independent identically distributed random experiments: I will call this the “iid hypothesis”. Now, suppose we observe option O₃, and let’s consider the specific observed sequence. Let r be the final observed proportion of As. This is some number strictly between 0 and 1 (otherwise we would have O₂). Of course, because we are in option O₃, we didn’t actually observe convergence to r, but still there was some final proportion. Whatever that number r is, van Inwagen thinks that if we had observed convergence to r, that would have been evidence against the agential control hypothesis, because it would have indicated a probabilistic random structure. But now observe that on the iid hypothesis, if there are m of the As and n of the Bs in a sequence, and the probability of A is p, then the probability of the sequence is p^m(1−p)ⁿ. Note that this does not depend on the order of the As and Bs in the sequence. Hence when we keep fixed the proportion of As in the sequence, any particular sequence that looks like it converges to that proportion and any particular sequence that simply happens, after some swings, to end at that proportion are equally likely. Further, the main competing hypothesis with the iid hypothesis is that no meaningful probabilities can be attached to the situation. In that case, we cannot say anything about whether O₁ or O₃ is more likely. Thus, any particular sequence we could get that exhibits O₁ does not favor the iid hypothesis any more than any particular sequence we could get that exhibits O₃.

Let us recapitulate. Option O₃ either is neutral on whether there is agential control or is some evidence against it. Option O₂ opposes agential control. Thus, option O₁ must either be neutral on whether there is agential control or be some evidence for agential control. And so van Inwagen’s replay argument does not work.

The Law of Large Numbers for independent identically distributed nonmeasurable random variables

Fact: For any real-valued function f, measurable or not, on a probability space, there exists a largest measurable function f^L such that f^L≤f and a smallest measurable function f^U such that f^U≥f, and f^L and f^U are unique up to almost sure equality.

Definition: A set U in a probability space is maximally nonmeasurable providing all its measurable subsets have measure zero and all its measurable supersets have measure one.

Definition: A sequence X₁,X₂,... of independent identically distributed not necessarily measurable random variables will be a sequence of functions on an infinite product of copies of a probability space, such that X_n(w₁,w₂,...)=F(w_n) for each n and a single fixed function F.

Henceforth suppose X₁,X₂,... are like that. Let S_n=X₁+...+X_n.

Easy consequence of the Law of Large Numbers: If X₁^L and X₁^U have finite expectations, then almost surely E[X₁^L]≤ liminf S_n/n≤ limsup S_n/n≤E[X₁^U].

Can one strengthen this? E.g., can one hope that one of the inequalities is an equality? Yesterday I finished proving a negative answer.

Theorem: Suppose X₁^L and X₁^U are integrable. Let A be any proper non-empty subset of the interval [E[X₁^L],E[X₁^U]] (which implies that E[X₁^L]<E[X₁^U]). Consider the respective subsets of our probability space where:

• lim S_n/n exists
• lim S_n/n exists and is in A
• limsup S_n/n is in A
• liminf S_n/n is in A
• all the limit points of S_n/n are in A

Then each of these subsets is maximally nonmeasurable.

This has a very interesting consequence for the philosophy of science, namely that unless we assume at the outset that what we are observing in the real world are measurable random variables, we can never come to that conclusion on the basis of observation of frequencies. For non-trivial cases (i.e., ones where E[X₁^L]<E[X₁^U]) of nonmeasurable random variables can equally well give neat limiting frequencies and not give them—any such limiting outcome is itself probabilistically maximally nonmeasurable.

Visits to a nonmeasurable set and a new sceptical worry

Let X₁,X₂,... be a sequence of independent, identically distributed random variables. Let H be a set of values, and let R_n(H) be the proportion of X₁,...,X_n that are in H. Thus R_n(H)=V_n(H)/n, where V_n(H) is the number of times that the sequence X₁,...,X_n has visited H. We can call R_n(H) the rate of visits to H.

The strong Law of Large Numbers then shows that if H is a measurable set, then, almost surely (i.e., with probability one), R_n(H) converges to P(X₁ in H). We can use X₁ (or any of the other variables, since they are identically distributed) to induce a probability measure P₀ on the set of possible values via the formula P₀(H)=P(X₁ in H). Thus, for measurable H, almost surely, lim R_n(H)=P₀(H). I.e., the asymptotic rate of visits to a measurable set H is equal to the probability of that set.

But what if H is nonmeasurable? We could consider the general case, but let's simplify and make things more interesting. What if H is maximally nonmeasurable? A set H is maximally nonmeasurable with respect to a probability measure P₀ if and only if:

• All the measurable subsets of H have measure zero.
• All the measurable supersets of H have measure one.

On a reasonable assignment of interval-valued probabilities, a maximally nonmeasurable set is one that gets the full interval [0,1].

Such sets are intuitively a mess. So what should we expect the rate of visits to a maximally nonmeasurable set to behave like. It was my intuition that we can expect the rate of visits to be a mess—to not converge to any particular value.[note 1] Here's a precise way to formulate the question. Let B be any non-empty proper subset of the interval [0,1]. Form the following subsets of our original P-probability space:

• L(H): the set of points of the probability space such that lim R_n(H) exists.
• L_B(H): the set of points of the probability space such that lim R_n(H) exists and falls in B.
• I_B(H): the set of points of the probability space such that liminf R_n(H) falls in B.
• S_B(H): the set of points of the probability space such that limsup R_n(H) falls in B.

My intuition that we should expect the rate of visits to be a nonconvergent mess is an intuition that L(H) should have high probability, or, if it is itself nonmeasurable, it should contain a measurable subset of high probability. If some proofs that I haven't checked all the details of are correct, this intuition is wrong.

Conjecture (Theorem if my proofs are right): The sets L(H), I_B(H), S_B(H) and L_B(H) are all maximally nonmeasurable if H is maximally nonmeasurable.

If this is correct, then there is basically nothing probabilistic you can say about the asymptotic convergence of R_n(H) for a maximally nonmeasurable set H. You can't say that the rate of visits probably will converge (no surprise there) and you can't say that it probably won't.

So what? Well, consider now a new sceptical problem. We perform some experiment E a thousand times, and 405 times we get outcome H. We very reasonably want to conclude that the circumstances of the experiment E have approximately a 40% tendency of producing outcome H. And the greater the number of experiments we do, as long as the observed rate of H's is around 40%, the more confident we are of this judgment, with our confidence going to one in the limit.

But wait! What about the sceptical hypothesis that the objective chances are such that H is maximally nonmeasurable given E? It is tempting to say: "Well, that could be true, but the longer our sequence of experiments with a rate of around 40%, the more confident we should be that H is measurable and has measure around 40%. We just wouldn't expect to get such nice convergence if H were maximally nonmeasurable." However, the Conjecture, assuming it's correct, shows that as a piece of probabilistic reasoning, this is completely wrong. For it is neither likely nor unlikely on the maximal nonmeasurability hypothesis that we would observe an asymptotic rate of 40% (or of any other value). To see this, let B be the singleton {0.4}, and note that L_B(H) is maximally nonmeasurable. Thus, its interval-valued probability is all of [0,1], and we can have no probabilistic expectations about it.

If the maximal nonmeasurability hypothesis cannot be ruled out a posteriori, and yet must be ruled out, then it must be ruled out a priori. I think our best hope is a postulate that outcomes are always at least partly measurable, i.e., aren't maximally nonmeasurable. And that's a kind of Principle of Sufficient Reason.

I think my (unchecked) proofs of the Conjecture can generalize to give a more complicated result in the case of nonmaximally measurable sets.

Attitudes to risk and the law of large numbers

People do things that seem to be irrational in respect of maximizing expected utilities. For instance, art collectors buy insurance, even though it seems that the expected payoff of buying insurance is negative—or else the insurance company wouldn't be selling it (some cases of insurance can be handled by distinguishing utilities from dollar amounts, as I do here, but I am inclined to think luxury items like art are not a case like that). Likewise, people buy lottery tickets, and choose the "wrong" option in the Allais Paradox.

Now, there are all sorts of clever decision-theoretic ways of modeling these phenomena and coming up with variations on utility-maximization that handle them. But rather than doing that I want to say something else about these cases.

Why is it good to maximize expected utilities in our choices (and let's bracket all deontic constraints here—let's suppose that none of the choices are deontically significant)? Well, a standard and plausible justification involves the Law of Large Numbers [LLN] (I actually wonder if we shouldn't be using the Central Limit Theorem instead—that might even strengthen the point I am going to make). Suppose you choose between option A and option B in a large number of independent trials. Then, on moderate assumptions on A and B, the LLN applies and says that if the number of trials N is large, probably the payoff for choosing A each time will be relatively close to NE[A] and the payoff for choosing B each time will be relatively close to NE[B], where E[A] and E[B] are the expected utilities of A and B, respectively. And so if E[A]>E[B], you will probably do better in the long run by choosing A rather than by choosing B, and you can (on moderate assumptions on A and B, again) make the probability that you will do better by choosing A as high as you like by making the number of trials large.

But here's the thing. My earthly life is finite (and I have no idea how decision theory is going to apply in the next life). I am not going to have an infinite number of trials. So how well this LLN-based argument works depends on how fast the convergence of observed average payoff to the statistically expected payoff in the LLN is. If the convergence is too slow relative to the expected number of A/B-type choices in my life, the argument is irrelevant. But now here's the kicker. The rate of convergence in the LLN depends on the shape of the distributions of A and B, and does so in such a way that the lop-sided distributions involved in the problems mentioned in the first paragraph of the paper are going to give particularly slow convergence. In other words, the standard LLN-based argument for expected utility maximization applies poorly precisely to the sorts of cases where people don't go for expected utility maximization.

That said, I don't actually think this cuts it as a justification of people's attitudes towards things like lotteries and insurance. Here is why. Take the case of lotteries. With a small number of repetitions, the observed average payoff of playing the lottery will likely be rather smaller than the expected value of the payoff, because the expected value of the payoff depends on winning, and probably you won't win with a small number of repetitions. So taking into account the deviation from the LLN actually disfavors playing the lottery. The same goes for insurance and Allais: taking into account the deviation from the LLN should, if anything, tell against insuring and choosing the "wrong" gamble in Allais.

Maybe there is a more complex explanation--but not justification--here. Maybe people sense (consciously or not—there might be some evolutionary mechanism here) that these cases don't play nice with the LLN, and so they don't do expected utility maximization, but do something heuristic, and the heuristic fails.