Epistemic utilities and decision theories

Warning: I worry there may be something wrong in the reasoning below.

Causal Decision Theory (CDT) and Epistemic Decision Theory (EDT) tend to disagree when the payoff of an option statistically depends on your propensity to go for that option. The most example of this phenomenon is Newcomb’s Problem (where money is literally put into a box or not depending on what your propensities are), and there is a large literature of other clever and mind-twisting examples. From the literature, one might get a feeling that these cases are all somehow weird, and normally there is no such dependence.

But here is a family of cases that happens literally almost all the time to us. Pretty much whenever we act we gain information relevant to facts about ourselves, and specifically to facts about our propensities to act. For instance, when you choose chocolate over vanilla ice cream you raise your credence for the hypothesis that you have a greater propensity to choose chocolate ice cream than to choose vanilla ice cream. But truth about oneself is valuable and falsehood about oneself is disvaluable. If in fact you have a greater propensity to choose chocolate ice cream, then by eating chocolate ice cream you gain credence in a truth, which is a good thing. If in fact your propensity for vanilla ice cream is at least as great as for chocolate ice cream, then by eating chocolate ice cream, you gain credence in a falsehood. The payoffs of your decision as to flavor of ice cream thus statistically depend on what your propensities actually are, and so this is exactly the kind of case where we would expect CDT and EDT to disagree.

Let’s be more precise. You have a choice between eating chocolate ice cream (C), eating vanilla ice cream (V) or not eating ice cream at all (N). Let H be the hypothesis that you have a greater propensity for eating chocolate ice cream than for eating vanilla ice cream. Then if you choose C, you will gain evidence for H. If you choose V, you will gain evidence for not-H. And if you choose N, you will (plausibly) gain no evidence for or against H. Your epistemic utility with respect to H is, let us suppose, measured by a single-proposition accuracy scoring rule, which we can think of as a pair of functions T_H and F_H, where T_H(p) is the value of having credence p in H if in fact H is true and F_H(p) is the value of having credence p in H if in fact H is false.

The expected evidential utilities of your three options are:

• E_e(C) = P(H|C)T_H(P(H|C)) + (1−P(H|C))F_H(P(H|C))

• E_e(V) = P(H|V)T_H(P(H|V)) + (1−P(H|V))F_H(P(H|V))

• E_e(N) = P(H|N)T_H(P(H|N)) + (1−P(H|N))F_H(P(H|N)) = P(H)T_H(P(H)) + (1−P(H))F_H(P(H)).

The expected causal utilities are:

• E_c(C) = P(H)T_H(P(H|C)) + (1−P(H))F_H(P(H|C))

• E_c(V) = P(H)T_H(P(H|V)) + (1−P(H))F_H(P(H|V))

• E_c(N) = P(H)T_H(P(H|N)) + (1−P(H))F_H(P(H|N)) = P(H)T_H(P(H)) + (1−P(H))F_H(P(H)).

We can make some quick observations in the case where the scoring rule is strictly proper, given that P(H|V) < P(H) < P(H|C):

1. E_c(C) < E_c(N)

2. E_c(V) < E_c(N)

3. At least one of E_e(C) > E_e(N) and E_e(V) > E_e(N) is true.

Observations 1 and 2 follow immediately from strict propriety and the formulas for E_c. Observation 3 follows from the fact that the expected accuracy score after Bayesian update on evidence is better (in non-trivial cases where the scoring rule is strictly proper) than before update, and the expected accuracy score after update on what you’ve chosen is:

• P(C)E_e(C) + P(V)E_e(V) + P(N)E_e(N)

while the expected accuracy score before update is equal to E_e(N). Since P(C) + P(V) + P(N) = 1, it follows from the superiority of the post-update expectation that at least one of E_e(C) and E_e(V) must be bigger than E_e(N).

The above results seem to be a black eye for CDT, which recommends that if what you care about is your epistemic utility with regard to your propensities regarding chocolate and vanilla ice cream, then you should always avoid eating ice cream!

(What about ratifiability? Some CDTers say that only ratifiable options should count. Is N ratifiable? Given that you’ve learned nothing about H from choosing N, I think N should be ratifiable. But I may be missing something. I find the epistemic utility case confusing.)

It also seems to me (I haven’t checked details) that on EDT there are cases where eating either flavor is good for you epistemically, but there are also cases where only one specific flavor is good for you.

An instability in Newcomb one-boxing

Consider Newcomb’s Paradox, and assume the predictor has a high accuracy but is nonetheless fallible. Suppose you have the character of a one-boxer and you know it. Then you also know that the predictor has predicted your choosing one box and hence you know that there is money in both boxes. It is now quite obvious that you should go for two boxes! Of course, like the predictor, you predict that you won’t do it. But there is nothing unusual about a situation where you predict you won’t do the rational thing: weakness of the will is a sadly common phenomenon. Similarly, if you have the character of a two-boxer and you know it, the rational thing to do is to go for two boxes. For in this case you know the predictor put money only in the clear box, and it would be stupid to just go for the opaque box and get nothing.

None of what I said above should be controversial. If you know what the predictor did, you should take both boxes. It’s like Drescher’s Transparent Newcomb Problem where the boxes are clear and it seems obvious you should take both. (That said, some do endorse one-boxing in Transparent Newcomb!) Though you should be sad that you are the sort of person who takes both.

This means that principles that lead to one-boxing suffer from an interesting instability: if you find out you are firmly committed to acting in accordance with these principles, it is irrational for you to act in accordance with them. Not so for the principles that lead to two-boxing. Even when you find out you are firmly committed to them, it’s rational to act in accordance with them.

This instability is a kind of flip of the usual observation that if one expects to be faced with Newcomb situations, and one has two-boxing principles, then it becomes rational to regret having one-boxing principles. That, too, is an odd kind of inconsistency. But this inconsistency does not seem particularly telling. Take any correct rational principle R. There are situations where it becomes rational to regret having R, e.g., if a madman is going around torturing all the people who have R. (This is similar to the example that Xenophon attributes to Socrates that being wise can harm you because it can lead to your being kidnapped by a tyrant to serve as his advisor.)

More decision theory stuff

Suppose there are two opaque boxes, A and B, of which I can choose one. A nearly perfect predictor of my actions put $100 in the box that they thought I would choose. Suppose I find myself with evidence that it’s 75% likely that I will choose box A (maybe in 75% of cases like this, people like me choose A). I then reason: “So, probably, the money is in box A”, and I take box A.

This reasoning is supported by causal decision theory. There are two causal hypotheses: that there is money in box A and that there is money in box B. Evidence that it’s 75% likely that I will choose box A provides me with evidence that it’s close to 75% likely that the predictor put the money in box A. The causal expected value of my choosing box A is thus around $75 and the causal expected value of my choosing box B is around $25.

On evidential decision theory, it’s a near toss-up what to do: the expected news value of my choosing A is close to $100 and so is that of my choosing B.

Thus, on causal decision theory, if I have to pay a $10 fee for choosing box A, while choosing box B is free, I should still go for box A. But on evidential decision theory, since it’s nearly certain that I’ll get a prize no matter what I do, it’s pointless to pay any fee. And that seems to be the right answer to me here. But evidential decision theory gives the clearly wrong answer in some other cases, such as that infamous counterfactual case where an undetected cancer would make you likely to smoke, with no causation in the other direction, and so on evidential decision theory you refrain from smoking to make sure you didn’t get the cancer.

In recent posts, I’ve been groping towards an alternative to both theories. The alternative depends on the idea of imagining looking at the options from the standpoint of causal decision theory after updating on the hypothesis that one has made a specific choice. In current my predictor cases, if you were to learn that you chose A, you would think: Very likely the money is in box A, so choosing box A was a good choice, while if you chose B, you would think: Very likely the money is in box B, so choosing box B was a good choice. As a result, it’s tempting to say that both choices are fine—they both ratify themselves, or something like that. But that misses out the plausible claim that if there is a $10 fee for choosing A, you should choose B. I don’t know how best to get that claim. Evidential decision theory gets it, but evidential decision theory has other problems.

Here’s something gerrymandered that might work for some binary choices. For options X and Y, which may or may not be the same, let e_X(Y) be the causal expected value of Y with respect to the credences for the causal hypotheses updated with respect to your having chosen X. Now, say that the differential restrospective causal expectation d(X) of option X equals e_X(X) − e_X(Y). This measures how much you would think you gained, from the standpoint of causal decision theory, in choosing X rather than Y by the lights of having updated on choosing X. Then you should the option that provides a bigger d(X).

In the case where there is a $10 fee for choosing box A, d(B) is approximately $100 while d(A) is approximately $90, so you should go for box B, as per my intuition. So you end up agreeing with evidential decision theory here.

You avoid the conclusion you should smoke to make sure you don’t have cancer in the hypothetical case where cancer causes smoking but not conversely, because the differential retrospective causal expectation of smoking is positive while the differential retrospective causal expectation of not smoking is negative, assuming smoking is fun (is it?). So here you agree with causal decision theory.

What about Newcomb’s paradox? If the clear box has a thousand dollars and the opaque box has a million or nothing (depending on whether you are predicted to take just the opaque box or to take both), then the differential retrospective causal expectation of two-boxing is a thousand dollars (when you learned you two-box, you learn that the opaque box was likely empty) and the differential retrospective causal expectation of one-boxing is minus a thousand dollars.

So the differential retrospective causal expectation theory agrees with causal decision theory in the clear case (cancer-causes-smoking), the difficult case (Newcomb), but agrees with evidential decision theory in the $10 fee variant of my two-box scenario, and the last seems plausible.

But (a) it’s gerrymandered and (b) I don’t know how to generalize it to cases with more than two options. I feel lost.

Maybe I should stop worrying about this stuff, because maybe there just is no good general way of making rational decisions in cases where there is probabilistic information available to you about how you will make your choice.

An immediate regret principle

Here’s a plausible immediate regret principle:

1. It is irrational to make a decision such that learning that you’ve made this decision immediately makes it rational to regret that you didn’t make a different decision.

The regret principle gives an argument for two-boxing in Newcomb’s Paradox, since if you go for one box, as soon as you have made your decision to do that, you will regret you didn’t make the two-box decision—there is that clear box with money staring at you, but if you go for two boxes, you will have no regrets.

Interestingly, though, one can come up with predictor stories where one has regrets no matter what one chooses. Suppose there are two opaque boxes, A and B, and you can take either box but not both. A predictor put a thousand dollars in the box that they predicted you won’t take. Their prediction need not be very good—all we need for the story is that there is a better than even probability of their having predicted you choosing A conditionally on your choosing A and a better than even probability of their having predicted you choosing B conditionally on your choosing B. But now as soon as you’ve made your decision, and before you opened the chosen box, you will think the other box is more likely to have the money, and so your knowledge of your decision will make it rational to regret that decision. Note that while the original Newcomb problem is science-fictional, there is nothing particularly science-fictional about my story. It would not be surprising, for instance, if someone were able to guess with better than even chance of correctness about what their friends would choose.

Is this a counterexample to the immediate regret principle (1), or is this an argument that there are real rational dilemmas, cases where all options are irrational?

I am not sure, but I am inclined to think that it’s a counterexample to the regret principle.

Can we modify the immediate regret principle to save it? Maybe. How about this?

2. No decision is such that learning that you’ve rationally made this decision immediately makes it rationally required to regret that you didn’t make a different decision.

On this regret principle, regret is compatible with non-irrational decision making but not with (known) rational decision making.

In my box story, it is neither rational nor irrational to choose A, and it is neither rational nor irrational to choose B. Then there is no contradiction to (2), since (2) only applies to decisions that are rationally made. And applying (2) to Newcomb’s Paradox no longer yields an argument for two-boxing, but only an argument that it is not rational to one-box. (For if it were rational to one-box, one could rationally decide to one-box, and one would then regret that.)

The “rationally” in (2) can be understood in a weaker way or a stronger way (the stronger way reads it as “out of rational requirement”). On either reading, (2) has some plausibility.

An odd decision theory

Suppose I am choosing between options A and B. Evidential decision theory tells me to calculate the expected utility E(U|A) given the news that I did A and the expected utility E(U|B) given the news that I did B, and go for the bigger of the two. This is well-known to lead to the following absurd result. Suppose there is a gene G that both causes one day to die a horrible death and makes one very likely to choose A, while absence of the gene makes one very likely to choose B. Then if A and B are different flavors of ice cream, I should always choose B, because E(U|A) ≪ E(U|B), since the horrible death from G trumps any advantage of flavor that A might have over B. This is silly, of course, because one’s choice does not affect whether one has G.

Causal decision theorists proceed as follows. We have a set of “causal hypotheses” about what the relevant parts of the world at the time of the decision are like. For each causal hypothesis H we calculate E(U|H∧A) and E(U|H∧B), and then we take the weighted average over our probabilities, and then decide accordingly. In other words, we have a causal expected utility of D

• E_c(U|D) = ∑_HE(U|H∧D)P(H)

and are to choose A over B provided that E_c(U|A) = E_c(U|B). In the gene case, the “bad news” of the horrible death on G is a constant addition to E_c(U|A) and to E_c(U|B), and so it can be ignored—as is right, since it’s not in our control.

But here is a variant case that worries me. Suppose that you are choosing between flavors A and B of ice cream, and you will only ever ever get to taste one of them, and only once. You can’t figure out which one will taste better for you (maybe one is oyster ice cream and the other is sea urchin ice cream). However, data shows that not only does G make one likely to choose A and its absence makes one likely to choose B, but everyone who has G derives pleasure from A and displeasure from B and everyone who lacks G has the opposite result, and all the pleasures and displeasures are of the same magnitude.

Now, background information says that you have a 3/4 chance of having G. On causal decision theory, this means that you should choose A, because likely you have G, and those who have G all enjoy A. Evidential decision theory, however, tells you that you should choose B, since if you choose B then likely you don’t have the terrible gene G.

In this case, I feel causal decision theory isn’t quite right. Suppose I choose A. Then after I have made my choice, but before I have consumed the ice cream, I will be glad that I chose A: my choice of A will make me think I have G, and hence that A is tastier. But similarly, if I choose B, then after I have made my choice, and again before consumption, I will be glad that I chose B, since my choice B will make me think I don’t have G and hence that B was a good choice. Whatever I choose, I will be glad I chose it. This suggests to me that my there is nothing wrong with either choice!

Here is the beginning of a third decision theory, then—one that is neither causal nor evidential. An option A is permissible provided that causal decision theory with the causal hypothesis credences conditioned on one’s choosing A permits one to do A. An option A is required provided that no alternative is permissible. (There are cases where no option is permissible. That’s weird, I admit.)

In the initial case, where the pleasure of each flavor does not depend on G, this third decision theory gives the same answer as causal decision theory—it says to go for the tastier flavor. In the second case, however, where the pleasure/displeasure depends on G, it permits one to go for either flavor. In a probabilistic-predictor Newcomb’s Paradox, it says to two-box.

Newcomb's Paradox and Pascal's Wager

Let Egalitarian Universalism (EU) be the doctrine that God exists and gives everyone infinite happiness, and that the quantity of this happiness is the same for everyone. The traditional formulation of Pascal's Wager obviously does not work in the case of the God of EU. What is surprising, however, is that one can make Pascal's Wager work even given the God of EU if one thinks that Bayesian decision theory, and hence one-boxing, is the right way to go in the case of Newcomb's Paradox with a not quite perfect predictor.

Here is how the trick works. Suppose that the only two epistemically available options are EU and atheism, and I need to decide whether or not to believe in God. Given Bayesian decision theory, I should choose whether to believe based on the conditional expected utilities. I need to calculate:

1. U₁=rP(EU|believe) + aP(atheism|believe)
2. U₂=rP(EU|~believe) + bP(atheism|~believe)

where r is the infinite positive reward that EU guarantees everybody, and a and b are the finite goods or bads of this life available if atheism is true. If U₁ is greater than U₂, then I should believe.

We'll need to use our favorite form of non-standard analysis for handling infinities. Observe that

3. P(believe|EU)>P(believe|~EU),

since a God would be moderately to want people to believe in him, and hence it is somewhat more likely that there would be theistic belief if God existed than if atheism were true (and I assumed that atheism and EU are the only options). But then by Bayes' Theorem it follows from (3) that:

4. P(EU|believe)>P(EU|~believe).

Let c=P(EU|believe)-P(EU|~believe). By (4), c is a positive number. Then:

5. U₁−U₂=rc + something finite.

Since r is infinite and positive, it follows that U₁−U₂>0, and hence U₁>U₂, so I should believe in EU.

The argument works on non-egalitarian universalism, too, as long as we don't think God gives an infinitely greater reward to those who don't believe in him.

However, universalism is false and one-boxing is mistaken.