Truth-directedness and propriety of scoring rules does not imply strict propriety

A scoring rule assigns a score to a credence assignment (which can but need not satisfy the axioms of probability), where a score is a random variable measuring how close the credence assignment is to the truth.

A scoring rule is strictly truth-directed provided that if c′ is a credence assignment that is closer to the truth than c is at ω, then c′ gets a better a score at ω. A scoring rule is proper provided that for all probabilities p, the p-expected value of the score of a probability p is at least as good as the p-expected value of the score of any other credence, and is strictly proper.

Propriety for a scoring rule is a pretty plausible condition, but it’s a bit harder to argue philosophically for strict propriety. But scoring-rule based philosophical arguments for probabilism—the doctrine that credences ought to be probabilities—require strict propriety.

In a clever move, Campbell-Moore and Levinstein showed that propriety plus strict truth-directedness and additivity (the idea that the score can be decomposed into a sum of single-event scores) implies strict propriety.

Here’s an interesting fact I will show: propriety plus strict truth-directedness do not imply strict propriety in the absence of additivity. Further, my counterexample will be bounded, infinitely differentiable and strictly proper on the probabilities. Personally don’t find additivity all that plausible, so I conclude the Campbell-Moore and Levinstein move does not move the discussion of strict propriety and probabilism ahead much.

Let Ω = {0, 1}. Given a credence function c (with values in [0,1]) on the powerset of Ω, define the credence function c^* which has the same value as c on the empty set and on Ω, but where c^*({0}) is the number z in [0,1] that minimizes (c({0})−z)² + (c({1})−(1−z))², and where c^*({1}) = 1 − c^*({0}). In other words, c^* is the credence function closest to c in the Euclidean metric such that c^*({0}) + c^*({1}) = 1.

Now let b^*(c) = b(c^*). Then b^* agrees with b score on the probabilities, and hence is strictly proper on them. Further, every value of b^* is a Brier score of some credence, and hence b^* is proper.

We now check that it is strictly truth-directed. Brier scores are strictly truth-directed. Thus, replacing a credence function with one that is closer to the truth on Ω or on the empty set will improve the b^* score. Moreover, it is easy to check that c^*({0}) = (1+c({0})−c({1}))/2. It’s easy to check that if we tweak c({0}) to move us closer to the truth at some fixed ω ∈ {0, 1}, then c^* will be closer to the truth at ω as well, and similarly if we tweak c({1}) to be closer to the truth at ω, and in both cases we will improve the score by the strict truth-directedness of Brier scores.

Finally, however, note that b^* is not strictly proper and does not have a domination theorem of the sort used in arguments for probabilism, since the b^*-score of any credence c that fails to be a probability due to its being the case c({0}) + c({1}) ≠ 1 but that gets the right values on the empty set and Ω (zero and one, respectively) is equal to the b^*-score of c^*, and c^* will be a probability in that case.

Note that in the example above we don't have quasi-strict propriety either.