The Law of Large Numbers and infinite run payoffs

In discussions of maximization of expected value, the Law of Large Numbers is sometimes invoked, at times—especially by me—off-handedly. According to the Strong Law of Large Numbers (SLLN), if you have an infinite sequence of independent random variables X₁, X₂, ... satisfying some conditions (e.g., in the Kolmogorov version ∑_n(σ_n²/n²) < ∞, where σ_n² is the variance of X_n), then with probability one, the average of the random variables converges to the average of the mathematical expectations of the random variables. The thought is that in that case, if the expectation of each X_n is positive, it is rationally required to accept the bet represented by X_n.

In a recent post, showed how in some cases where the Strong Law of Large Numbers is not met, in an infinite run it can be disastrous to bet in each case according to expected value.

Here I want to make a minor observation. The fact that the SLLN applies to some sequence of independent random variables is itself not sufficient to make it rational to bet in each case according to the expectations in an infinite run. Let X_n be 2ⁿ/n with probability 1/2ⁿ and  − 1/(2n) with probability 1 − 1/2ⁿ. Then

• EX_n = (1/2ⁿ)(2ⁿ/n) − 1/(2n)(1−1/2ⁿ) = (1/n)(1−(1/2)(1−1/2ⁿ)).

Clearly EX_n > 0. So in individual decisions based on expected value, each X_n will be a required bet.

Now, just as in my previous post, almost surely (i.e., with probability one) only finitely many of the bets X_n will have the positive payoff. Thus, with a finite number of exceptions, our sequence of payoffs will be the sequence  − 1/2,  − 1/4,  − 1/6,  − 1/8, .... Therefore, almost surely, the average of the first n payoffs converges to zero. Moreover, the average of the first n mathematical expectations converges to zero. Hence the variables X₁, X₂, ... satisfy the Strong Law of Large Numbers. But what is the infinite run payoff of accepting all the bets? Well, given that almost surely there are only a finite number of n such that the payoff of bet n is not of the form  − 1/(2n), it follows that almost surely the infinite run payoff differs by a finite amount from  − 1/2 − 1/4 − 1/6 − 1/8 =  − ∞. Thus the infinite run payoff is negative infinity, a disaster.

Hence even when the SLLN applies, we can have cases where almost surely there are only finitely many positive payments, infinitely many negative ones, and the negative ones add up to  − ∞.

In the above example, while the variables satisfy the SLLN, they do not satisfy the conditions for the Kolmogorov version of the SLLN: the variances grows exponentially. It is somewhat interesting to ask if the variance condition in the Kolmogorov Law is enough to prevent this pathology. It’s not. Generalize my example by supposing that a₁, a₂, ... is a sequence of numbers strictly between 0 and 1 with finite sum. Let X_n be 1/(na_n) with probability a_n and  − 1/(2n) with probability 1 − a_n. As before, the expected value is positive, and by Borel-Cantelli (given that the sum of the a_n is finite) almost surely the payoffs are  − 1/(2n) with finitely many exceptions, and hence the there is a finite positive payoff and an infinite negative one in the infinite run.

But the variance σ_n² is less than a_n/(na_n)² + 1 = (1/(n²a_n)) + 1. If we let a_n = 1/n² (the sum of these is finite), then each variance is at most 2, and so the conditions of the Kolmogorov version of the SLLN are satisfied.

In an earlier post, I suggested that perhaps the Central Limit Theorem (CLT) rather than the Law of Large Numbers is what one should use to justify betting according to expected utilities. If the variables X₁, X₂, ... satisfy the conditions of the CLT, and have non-negative expectations, then P(X₁+...+X_n≥0) will eventually exceed any number less than 1/2. In particular, we won’t have the kind of disastrous situation where the overall payoffs almost surely go negative, and so no example like my above one can satisfy the conditions of the CLT.