A scoring rule s assigns to a credence c on a space Ω a score s(c) measuring the inaccuracy of c. The score is itself a function from Ω to [−∞,M] (for some fixed M), so that s(c)(ω) measures the inaccuracy of c if the truth of the matter is that we are at ω ∈ Ω. A scoring rule is proper provided that Eps(p) ≤ Eps(c) for any probability p and any other credence c: i.e., provided that the p-expected value of the score of p is at least as good as (no more inaccurate than) as the p-expected value of any other score. It is strictly proper if the inequality is strict. It is quasi-strictly proper if the inequality is strict whenever p is a probability and c is not.
Here’s a fun fact about scoring rules.
Theorem: Let s be any continuous bounded proper scoring rule that is defined only on the probabilities. Then s can be extended to a continuous bounded quasi-strictly proper scoring rule defined on all credences.
This result works in the finite-dimensional case with standard Euclidean topologies on the space of credences (considered as elements of [0,1]PΩ) and on the scores (considered as values in [−∞,M]Ω). But it also works in countably-infinite-dimensional contexts in the right topologies (ℓ∞(PΩ) on the side of the credences and the product topology on the side of the scores), regardless of whether by “probabilities” we mean finitely or countably additive ones.
The proof uses three steps.
First, show that the probabilities are a closed subset of the space of credences.
Second, apply the Dugundji extension theorem to extend the score s from the probabilities to all credences while maintaining continuity and ensuring that the range of the extension is a subset of the convex hull of the range of the original score. Let s0 be the extended score. The convex hull condition and the propriety of the original score on the probabilities implies that s0 is proper, though not necessarily quasi-strictly so.
Third, let s1(c) = d(c,P) + s0(c), where Q is the set of credences that are probabilities and d(c,Q) is the distance from c to Q in the ℓ∞(PΩ) norm. This equals s0(c) and hence s(c) for a credence c.
I don’t know if there is much of a philosophical upshot of this. Maybe a kind of interesting upshot is that it illustrates that quasi-strict propriety is easy to generate?
4 comments:
Not really relevant, but I had literally just been reading Philip Dawid on proper scoring rules and how they relate to composite likelihood but also robust methods (https://doi.org/10.1007/s40300-014-0039-y). The other paper
https://arxiv.org/abs/1409.5291
discusses using them to do Bayesian model selection. Is there some way to carry this kind of thing across to credences?
I think that scoring rules for non-probability credences are either ad hoc or a spandrel, except maybe if we assume additivity.
BTW, in Step 2 we can make s0 have the same range as s if we want: just apply the Dugundji extension theorem to the identity function on the set of all probabilities (which is a convex set) to get a continuous function f from credences to probabilities that is equal to identity on the probabilities. Then let s0(c)=s(f(c)).
Post a Comment