Consider this line of reasoning:
- Given my evidence, I should do A rather than B.
- So, given my evidence, it is likely that A will be better than B.
- The expected utility of A is higher than that of B.
- So, probably, A has higher utility than B.
But even when there is a fact of the matter, the inference from (3) to (4) is fallacious, due to simple cases. Suppose that a die has been rolled but I haven't seen the result. I can choose to play game A which pays $1000 if the die shows 1 and nothing otherwise, or I have option B which is just to get a dollar no matter what. Then the expected utility of A is about $167 (think 1000/6) and the expected utility of B is exactly $1. However, there is a 5/6 chance that B has higher utility.
The lesson here is that our decisions are made on the basis of expected utilities rather than on the basis of the probabilities of the better outcome.
Now the application. One objection to some resolutions to the problem of evil, notably sceptical theism, is this line of thought:
- We are obligated to prevent evil E.
- So, probably, evil E is not outweighed by goods.
One might worry that the case of responses to the problem of evil isn't going to look anything like the cases that provide counterexamples to the expectation-probability fallacy. In other words, even though the expectation-probability fallacy is a fallacy in most cases, it isn't fallacious in the case of (5) and (6). But it's possible to provide a counterexample to the fallacy that is quite close to the sceptical theism case.
At this point the post turns a little more technical, and I won't be offended if you stop reading. Imagine that a quarter has been tossed a thousand times and so has a dime. There is now a game. You choose which coin counts—the quarter or the time—and then sequentially over the next thousand days you get a dollar for each heads toss and pay a dollar for each tails toss. Moreover, it is revealed to you that the first time the quarter was tossed it landed heads, while the first time the dime was tossed it landed tails.
It is clear that you should choose to base the game on the tosses of the quarter. For the expected utility of the first toss in this game is $1, and the expected utility of each subsequent toss is $0, for a total expected utility of one dollar, whereas the expected utility of the first toss in the dime-based game is $(-1), and the subsequent tosses have zero expected utility, so the expected utility is negative one dollar.
On the other hand, the probability that the quarter game is better than the dime game is insignificantly higher than 1/2. (We could use the binomial distribution to say just how much higher than 1/2 it is.) The reason for that is that the 999 subsequent tosses are very likely to swamp the result from the first toss.
Suppose now that you observe Godot choosing to play the dime game. Do you have significant evidence against the hypothesis that Godot is an omniscient self-interested agent? No. For if Godot is an omniscient self-interested agent, he will know how all the 1000 tosses of each coin went, and there is probability that's insignificantly short of 1/2 that they went in such a way that the dime game pays better.