Monday, December 13, 2021

Truth directed scoring rules on an infinite space

A credence assignment c on a space Ω of situations is a function from the powerset of Ω to [0, 1], with c(E) representing one’s degree of belief in E ⊆ Ω.

An accuracy scoring rule s assigns to a credence assignment c on a space Ω and situation ω the epistemic utility s(c)(ω) of having credence assignment c when in truth we are in ω. Epistemic utilities are extended real numbers.

The scoring rule is strictly truth directed provided that if credence assignment c2 is strictly truer than c1 at ω, then s(c2)(ω)>s(c1)(ω). We say that c2 is strictly truer than c1 if and only if for every event E that happens at ω, c2(E)≥c1(E) and for every event E that does not happen at ω, c2(E)≤c1(E), and in at least one case there is strict inequality.

A credence assignment c is extreme provided that c(E) is 0 or 1 for every E.

Proposition. If the probability space Ω is infinite, then there is no strictly truth directed scoring rule defined for all credences, or even for all extreme credences.

In fact, there is not even a scoring rule that strictly truth directed when restricted to extreme credences, where an extreme credence is one that assigns 0 or 1 to every event.

This proposition uses the following result that my colleague Daniel Herden essentially gave me a proof of:

Lemma. If PX is the power set of X, then there is no function f : PX → X such that f(A)≠f(B) whenever A ⊂ B.

Now, we prove the Proposition. Fix ω ∈ Ω. Let s be a strictly truth directed scoring rule defined for all extreme credences. For any subset A of PΩ, define cA to be the extreme credence function that is correct at ω at all and only the events in A, i.e., cA(E)=1 if and only if ω ∈ E and E ∈ A or ω ∉ E and E ∉ A, and otherwise cA(E)=0. Note that cB is strictly truer than cA if and only if A ⊂ B. For any subset A of PΩ, let f(A)=s(cA)(ω).

Then f(A)<f(B) whenever A ⊂ B. Hence f is a strictly monotonic function from PPΩ to the reals. Now, if Ω is infinite, then the reals can be embedded in PΩ (by the axiom of countable choice, Ω contains a countably infinite subset, and hence PΩ has cardinality at least that of the continuum). Hence we have a function like the one the Lemma denies the existence of, a contradiction.

Note: This suggests that if we want strict truth directedness of a scoring rule, the scoring rule had better take values in a set whose cardinality is greater than that of the continuum, e.g., the hyperreals.

Proof of Lemma (essentially due to Daniel Herden): Suppose we have f as in the statement of the Lemma. Let ON be the class of ordinals. Define a function F : ON → A by transfinite induction:

  • F(0)=f(⌀)

  • F(α)=f({F(β):β < α}) whenever α is a successor or limit ordinal.

I claim that this function is one-to-one.

Let Hα = {F(δ):δ < α}.

Suppose F is one-to-one on β for all β < α. If α is a limit ordinal, then it follows that F is one-to-one on α. Suppose instead that α is a successor of β. I claim that F is one-to-one on α, too. The only possible failure of injectivity on α could be if F(β)=F(γ) for some γ < β. Now, F(β)=f(Hβ) and F(γ)=f(Hγ). Note that Hγ ⊂ Hβ since F is one-to-one on β. Hence f(Hβ)≠f(Hγ) by the assumption of the Lemma. So, F is one-to-one on ON by transfinite induction.

But of course we can’t embed ON in a set (Burali-Forti).

1 comment:

  1. I shouldn't have said "the hyperreals", since the most common construction has the cardinality of the continuum (I am grateful to Snow Zhang for pointing this out to me) but "an appropriately constructed hyperreal field".

    Here's something like the construction I had in mind: Let F be the set of finite fields on Omega ordered by inclusion. Given G in F, let s_G(c)(omega) be the Brier accuracy score for c|G (c restricted to G) at omega. I.e., s_G(c)(omega) = sum_{A in G} -(1_A(omega)-c(A))^2.

    Let U be an ultrafilter on F. Let the hyperreal field be the ultraproduct of the reals over U, and let the score of a credence c at omega be the ultra product of s_G(c)(omega).

    Since s_G is truth-directed over G, s is truth-directed.

    A difficulty with hyperreal scores on an infinite space is that we don't have a straightforward definition of expected value and hence of propriety.

    ReplyDelete