One might think that being closer to the truth is guaranteed to get one to make better decisions. Not so. Say that a probability assignment p2 is at least as true as a probability assignment p1 at a world or situation ω provided that for every event E holding at ω we have p2(E)≥p1(E) and for every event E not holding at ω we have p2(E)≤p1(E). And say that p2 is truer than p1 provided that strict inequality holds in at least one case.
Suppose that a secret integer has been picked among 1, 2 and 3, and p1 assigns the respective probabilities 0.5, 0.3, 0.2 to the three possibilities while p2 assigns them 0.7, 0.1, 0.2. Then if the true situation is 1, it is easy to check that p2 is truer than p1. But now suppose that you are offered a choice between the following games:
W1: on 1 win $2, on 2 win $1100, and on 3 win $1000.
W2: on 1 win $1, on 2 win $1000, and on 3 win $1100
If you are going by p1, you will choose W1 and if you are going by p2, you will choose W2. But if the true number is 1, you would be better off picking W1 (getting $2 instead of $1), so the truer probabilities will lead to a worse payoff. C’est la vie.
Say that a scoring rule for probabilities is truth-directed if it never assigns a poorer score for a truer set of probabilities. The above example shows that a proper scoring rule need not be truth-directed. For let s(p)(n) be the payoff you will get if the secret number is n and you make your decision between W1 and W2 rationally on the basis of probability assignment p (with ties broken in favor of W1, say). Then s is a proper (accuracy) scoring rule but the above considerations show that s(p2)(1)<s(p1)(1), even though p2 is truer at 1. In fact, we can get a strictly proper scoring rule that isn’t truth-directed if we want: just add a tiny multiple of a Brier accuracy score to s.
Intuitively we would want our scoring rules to be both proper and truth-directed. But given that sometimes we are pragmatically better off for having less true probabilities, it is not clear that scoring rules should be truth-directed. I find myself of divided mind in this regard.
How common is this phenomenon? Roughly it happens whenever the truer and less-true probabilities disagree on ratios of probabilities of non-actual events.
Proposition: Suppose two probability assignments are such that there are events E1 and E2 with probabilities strictly between 0 and 1, with ω1 in neither event, and such that the ratio p1(E1)/p1(E2) is different from the ratio p2(E1)/p2(E2). Then there are wagers W1 and W2 such that p1 prefers W1 and p2 prefers W2, but W1 pays better than W2 at ω1.
1 comment:
Well, if it is that hard and difficult to go for the "truth", then you might as well go for the utility.
The expected utility with plus minus standard deviation for the first game W1 is
(531±530.13)$ and
The expected utility with plus minus standard deviation for the second game W2 is
(320.2±489.03)$.
Given this then I would tend more towards the first game W1, since the standard deviations are about the same for these two games, but the expected value for game W1 is significantly better than for the second game W2.
Post a Comment