Hyperreal expected value

I think I have a hyperreal solution, not entirely satisfactory, to three problems.

1. The problem of how to value the St Petersburg paradox. The particular version that interests me is one from Russell and Isaacs which says that any finite value is too small, but any infinite value violates strict dominance (since, no matter what, the payoff will be less than infinity).

2. How to value gambles on a countably infinite fair lottery where the gamble is positive and asymptotically approaches zero at infinity. The problem is that any positive non-infinitesimal value is too big and any infinitesimal value violates strict dominance.

3. How to evaluate expected utilities of gambles whose values are hyperreal, where the probabilities may be real or hyperreal, which I raise in Section 4.2 of my paper on accuracy in infinite domains.

The apparent solution works as follows. For any gamble with values in some real or hyperreal field V and any finitely-additive probability p with values in V, we generate a hyperreal expected value E_p, which satisfies these plausible axioms:

4. Linearity: E_p(af+bg) = aE_pf + bE_pg for a and b in V

5. Probability-match: E_p1_A = p(A) for any event A, where 1_A is 1 on A and 0 elsewhere

6. Dominance: if f ≤ g everywhere, then E_pf ≤ E_pg, and if f < g everywhere, then E_pf < E_pg.

How does this get around the arguments I link to in (1) and (2) that seem to say that this can’t be done? The trick is this: the expected value has values in a hyperreal field W which will be larger than V, while (4)–(6) only hold for gambles with values in V. The idea is that we distinguish between what one might call primary values, which are particular goods in the world, and what one might call distribution values, which specify how much a random distribution of primary values is worth. We do not allow the distribution values themselves to be the values of a gamble. This has some downsides, but at least we can have (4)–(6) on all gambles.

How is this trick done?

I think like this. First it looks like the Hahn-Banach dominated extension theorem holds for V₂-valued V₁-linear functionals on V₁-vector spaces V₁ ⊆ V₂ are real or hyperreal field, except that our extending functional may need to take values in a field of hyperreals even larger than V₂. The crucial thing to note is that any subset of a real or hyperreal field has a supremum in a larger hyperreal field. Then where the proof of the Hahn-Banach theorem uses infima and suprema, you move to a larger hyperreal field to get them.

Now, embed V in a hyperreal field V₂ that contains a supremum for every subset of V, and embed V₂ in V₃ which has a supremum for every subset of V₂. Let Ω be our probability space.

Let X be the space of bounded V₂-valued functions on Ω and let M ⊆ X be the subspace of simple functions (with respect to the algebra of sets that Ω is defined on). For f ∈ M, let ϕ(f) be the integral of f with respect to p, defined in the obvious way. The supremum on V₂ (which has values in V₃) is then a seminorm dominating ϕ. Extend ϕ to a V-linear function ϕ on X dominated by V₂. Note that if f > 0 everywhere for f with values in V, then f > α > 0 everywhere for some α ∈ V₂, and hence ϕ(−f) ≤  − α by seminorm domination, hence 0 < α ≤ ϕ(f). Letting E_p be ϕ restricted to the V-valued functions, our construction is complete.

I should check all the details at some point, but not today.

The Law of Large Numbers and infinite run payoffs

In discussions of maximization of expected value, the Law of Large Numbers is sometimes invoked, at times—especially by me—off-handedly. According to the Strong Law of Large Numbers (SLLN), if you have an infinite sequence of independent random variables X₁, X₂, ... satisfying some conditions (e.g., in the Kolmogorov version ∑_n(σ_n²/n²) < ∞, where σ_n² is the variance of X_n), then with probability one, the average of the random variables converges to the average of the mathematical expectations of the random variables. The thought is that in that case, if the expectation of each X_n is positive, it is rationally required to accept the bet represented by X_n.

In a recent post, showed how in some cases where the Strong Law of Large Numbers is not met, in an infinite run it can be disastrous to bet in each case according to expected value.

Here I want to make a minor observation. The fact that the SLLN applies to some sequence of independent random variables is itself not sufficient to make it rational to bet in each case according to the expectations in an infinite run. Let X_n be 2ⁿ/n with probability 1/2ⁿ and  − 1/(2n) with probability 1 − 1/2ⁿ. Then

• EX_n = (1/2ⁿ)(2ⁿ/n) − 1/(2n)(1−1/2ⁿ) = (1/n)(1−(1/2)(1−1/2ⁿ)).

Clearly EX_n > 0. So in individual decisions based on expected value, each X_n will be a required bet.

Now, just as in my previous post, almost surely (i.e., with probability one) only finitely many of the bets X_n will have the positive payoff. Thus, with a finite number of exceptions, our sequence of payoffs will be the sequence  − 1/2,  − 1/4,  − 1/6,  − 1/8, .... Therefore, almost surely, the average of the first n payoffs converges to zero. Moreover, the average of the first n mathematical expectations converges to zero. Hence the variables X₁, X₂, ... satisfy the Strong Law of Large Numbers. But what is the infinite run payoff of accepting all the bets? Well, given that almost surely there are only a finite number of n such that the payoff of bet n is not of the form  − 1/(2n), it follows that almost surely the infinite run payoff differs by a finite amount from  − 1/2 − 1/4 − 1/6 − 1/8 =  − ∞. Thus the infinite run payoff is negative infinity, a disaster.

Hence even when the SLLN applies, we can have cases where almost surely there are only finitely many positive payments, infinitely many negative ones, and the negative ones add up to  − ∞.

In the above example, while the variables satisfy the SLLN, they do not satisfy the conditions for the Kolmogorov version of the SLLN: the variances grows exponentially. It is somewhat interesting to ask if the variance condition in the Kolmogorov Law is enough to prevent this pathology. It’s not. Generalize my example by supposing that a₁, a₂, ... is a sequence of numbers strictly between 0 and 1 with finite sum. Let X_n be 1/(na_n) with probability a_n and  − 1/(2n) with probability 1 − a_n. As before, the expected value is positive, and by Borel-Cantelli (given that the sum of the a_n is finite) almost surely the payoffs are  − 1/(2n) with finitely many exceptions, and hence the there is a finite positive payoff and an infinite negative one in the infinite run.

But the variance σ_n² is less than a_n/(na_n)² + 1 = (1/(n²a_n)) + 1. If we let a_n = 1/n² (the sum of these is finite), then each variance is at most 2, and so the conditions of the Kolmogorov version of the SLLN are satisfied.

In an earlier post, I suggested that perhaps the Central Limit Theorem (CLT) rather than the Law of Large Numbers is what one should use to justify betting according to expected utilities. If the variables X₁, X₂, ... satisfy the conditions of the CLT, and have non-negative expectations, then P(X₁+...+X_n≥0) will eventually exceed any number less than 1/2. In particular, we won’t have the kind of disastrous situation where the overall payoffs almost surely go negative, and so no example like my above one can satisfy the conditions of the CLT.

Valuations and credences

One picture of credences is that they are derived from agents’ valuations of wagers (i.e., previsions) as follows: the agent’s credence in a proposition p is equal to the agent’s valuation of a gamble that pays one unit if p is true and 0 units if p false.

While this may give the right answer for a rational agent, it does not work for an irrational agent. Here are two closely related problems. First, note that the above definition of credences is dependent on the unit system in which the gambles are denominated. A rational agent who values a gamble that pays one dollars on heads and zero dollars otherwise at half a dollar will also value a gamble that pays one yen on heads and zero yen otherwise at half a yen, and we can attribute a credence of 1/2 in heads to the agent. In general, the rational agent’s valuations will be invariant under affine transformations and so we do not have a problem. But Bob, an irrational agent, might value the first gamble at $0.60 and the second at 0.30 yen. What, then, is that agent’s credence in heads?

If there were a privileged unit system for utilities, we could use that, and equate an agent’s credence in p with their valuation of a wager that pays one privileged unit on p and zero on not-p. But there are many units of utility, none of them privileged: dollars, yen, hours of rock climbing, glazed donuts, etc.

And even if there were a privileged unit system, there is a second problem. Suppose Alice is an irrational agent. Suppose Alice has two different probability functions, P and Q. When Alice needs to calculate the value of a gamble that pays exactly one unit on some proposition and exactly zero units on the negation of that proposition, she uses classical mathematical expectation based on P. When Alice needs to calculate the value of any other gamble—i.e., a gamble that has fewer than or more than two possible payoffs or a gamble that has two payoffs but at values other than exactly one or zero—she uses classical mathematical expectation based on Q.

Then the proposed procedure attributes to Alice the credence function P. But it is in fact Q that is predictive of Alice’s behavior. For we are never in practice offered gambles that have exactly two payoffs. Coin-toss games are rare in real life, and even they have more than two payoffs. For instance, suppose I tell you that I will give you a dollar on heads and zero otherwise. Well, a dollar is worth a different amount depending on when exactly I give it to you: a dollar given earlier is typically more valuable, since you can invest it for longer. And it’s random when exactly I will pay you. So on heads, there are actually infinitely many possible payoffs, some slightly larger than others. Moreover, there is a slight chance of the coin landing on the edge. While that eventuality is extremely unlikely, it has a payoff that’s likely to be more than a dollar: if you ever see a coin landing on edge, you will get pleasure out of telling your friends about it afterwards. Moreover, even if we were offered a gamble that had exactly two payoffs, it is extremely unlikely that these payoffs would be exactly one and zero in the privileged unit system.

The above cases do not undercut a more sophisticated story about the relationship between credences and valuations, a story on which one counts as having the credence that would best fit one’s practical valuations of gambles with two-values, and where there is a tie, one’s credences are underdetermined or interval-valued. In Alice’s case, for instance, it is easy to say that Q best fits the credences, while in Bob’s case, the credence for heads might be a range from 0.3 to 0.6.

But we can imagine a variant of Alice where she uses P whenever she has a gamble that has only two payoffs, and she uses Q at all other times. Since in practice two-payoff gambles don’t occur, she always uses Q. But if we use two-payoff gambles to define credences, then Alice will get P attributed to her as her credences, despite her never using P.

Can we have a more sophisticated story that allows credences to be defined in terms of valuations of gambles with more payoffs than two? I doubt it. For there are multiple ways of relating a prevision to a credence when we are dealing with an inconsistent agent, and none of them seem privileged. Even my favorite way, the Level Set Integral, comes in two versions: the Split and Shifted versions.

Proper scoring rules and gambling

There are two ways of evaluating a credence assignment. There is the decision-theoretic method: you consider how you are going to do given this credence assignment when presented with some gambles. And there is the scoring rule method: you consider how far you are from "the truth", i.e., the credence assignment that assigns 0 to the falsehoods and 1 to the truths, and you measure this with respect to a proper scoring rule.

There are various parallel results for the two methods.

It turns out that there is a good reason why there are parallel results. The two methods are equivalent. Assume an underlying probability space Ω. To avoid measurability issues, suppose Ω is finite. Denote a scoring rule by a function s(p,q) where p is a consistent credence assignment for some family of sentences and q is a consistent extreme credence assignment (0/1 valued) for the same family. By "the truth", I mean the extreme credence assignment that assigns 1 to each true sentence and 0 to each false one. A scoring rule is proper provided that E_p(−s(p',T))≤E_p(−s(p,T)) where T is the random variable that assigns to each point ω of Ω a function T(ω) that in turn assigns to each sentence its truth value at ω (i.e., 1 if true, 0 if false), and where E_p is expectation with respect to the credence assignment p.

Theorem. For any proper scoring rule s(p,q), there exists a family F of gambles such that for any consistent credence assignment there is a gamble that maximizes the expected payoff, and when you choose that maximizing gamble your payoff will be −s(p,T). Conversely, suppose that F is a family of gambles such that for any credence assignment there is a gamble that maximizes the expected payoff. Let V(p,q) be the payoff of such a gamble for credence assignment p when q is the truth. Then −V(p,q) is a proper scoring rule.

The proof is actually very simple. (I had very complicated proofs of special cases of this Theorem in the past, but now I see it is all very simple, even trivial.) For the left-to-right direction, for any possible credence assignment p, define the gamble G_p as follows: at ω, you get paid −s(p,T(ω)). Then the propriety of the scoring rule guarantees that G_p maximizes the expected payoff when p is your credence assigment. Conversely, let s(p,q)=−V(p,q). Propriety is easy to check—it just follows from maximization.

An argument for expected utility maximization

Until very recently, I thought there was only one argument for the idea that, barring deontic concerns and the like, rationality is connected to the maximization of expected utilities, namely the long-run advantage argument based on the Law of Large Numbers. But there is another: an argument from a plausible set of axioms for rational preferability. Fix a probability space Ω. Say that a gamble is a bounded real-valued random variable on Ω. Suppose that there is a rational preferability ordering < on gambles, where we write A<B if B is preferable to A. Here are some plausible axioms for <:

1. Transitivity: < is transitive
2. Domination: If A(ω)≤B(ω) for all ω∈Ω, then for all C, if C<A, then C<B, and if B<C, then A<C.
3. Sure Thing: If A and B are gambles that have certainty of getting payoffs a and b respectively, with a<b, then A<B.
4. Additivity: If A<C and B<D, then A+B<C+D.
5. Equivalence: If A and B are probabilistically equivalent (i.e., P(A∈U)=P(B∈U) for every measurable U), then for all C we have A<C if and only if B<C, and C<A if and only if C<B.

The most controversial will be, I suppose, Additivity. But there is a very simple argument for it: If you should choose C over A, and D over B, then you should choose the combination of C and D over the combination of A and B.

Add a handy technical assumption:

6. Continuity: There is a collection of events E_a, for 0<a<1, such that P(E_a)=a and E_a is a subset of E_b when a<b.

To get Continuity, all we need to do is suppose we've got some irrelevant continuous random process going on in our probability space, like the decay of an radioactive sample, or else suppose that we've got an infinite sequence of independent identically distributed coin flips, etc. Even if our world doesn't contain such a process, surely the same preferences would be rational in a world where some irrelevant-to-us such process takes place. So we can assume Continuity.

Theorem: Assume (1)-(6). If E(A)<E(B) for gambles A and B, then A<B.

The proof is given in this footnote: [note 1].

Personally, I am suspicious of transitivity in general, but I am less suspicious of it in the case of real-valued bounded gambles.

A stupid way to invest

Here's a fun little puzzle for introducing some issues in decision theory. You want to invest a sum of money that is very large for you (maybe it represents all your present savings, and you are unlikely to save that amount again), but not large enough to perceptibly affect the market. A reliable financial advisor suggests you diversifiedly invest in n different stocks, s₁,...,s_n, putting x_i dollars in s_i. You think to yourself: "That's a lot of trouble. Here is a simpler solution that has the same expected monetary value, and is less work. I will choose a random number j between 1 and n, such that the probability of choosing j=i is proportional to x_i (i.e., P(j=i)=x_i/(x₁+...+x_n)). Then I will put all my money in s_j." It's easy to check that this method does have the same expected value as the diversified strategy. But it's obvious that this is a stupid way to invest. The puzzle is: Why is this stupid?

Well, one standard answer is this. This is stupid because utility is not proportional to dollar amount. If the sum of money is large for you, then the disutility of losing everything is greater than the utility of doubling your investment. If that doesn't satisfy, then the second standard answer is that this is an argument for why we ought to be risk averse.

Maybe these answers are good. I don't have an argument that they're not. But there is another thought that from time to time I wonder about. We're talking of what is for you a very large sum of money. Now, the justification for expected-utility maximization is that in the long run it pays. But here we are dealing with what is most likely a one-time decision. So maybe the fact that in the long run it pays to use the simpler randomized investment strategy is irrelevant. If you expected to make such investments often, the simpler strategy would, indeed, be the better one—and would eventually result in a diversified portfolio. But for a one-time decision, things may be quite different. If so, this is interesting—it endangers Pascal's Wager, for instance.