Scoring rules measures the inaccuracy of one's credences. Roughly, when p is true, and one assigns credence r to p, then a scoring rule measures the distance between r and 1, while when p is false, the scoring rule measures the distance between r and 0. The smaller the score, the better.
Some scoring rules are better than others. Let's suppose some scoring rules are right. Then this thesis seems to be implicit in some applications of scoring rules (e.g., here):
- If S is the right scoring rule, then a credence-assignment policy is epistemically rational only if following the policy minimizes expected total or average S-scores.
But (1) is false. Here's a simple counterexample that works for most reasonable scoring rules. Consider a situation like this: A fair coin is flipped. If you assign credence 0.51 to heads, a mindreader who knows your credence assignments will immediately reveal to you how the coin landed. Otherwise, you will never have any information on how the coin landed.
Obviously, the epistemically rational thing to do is to assign 0.5 to heads. But this leads to higher expected total and average scores on most reasonable scoring rules. For if you assign 0.51, then once the mindreader tells you how the coin landed, you will update your credence to be very close to 0 or 1, and your score will be very low. And the only cost of this scenario is the slight inoptimality from briefly having score 0.51 instead of the optimal score of 0.5. So the epistemically rational policy for dealing with situations like this, namely assigning 0.5, does less well in expected scores than the epistemically irrational policy of assigning 0.51.
The case may seem farfetched. But there are real-life cases that may be similar. It may be that for psychological reasons when you are a bit more sure, or a bit less sure (depending on your character and the thesis), of a thesis than rationality calls for, you will be better able to investigate whether the thesis is true. Thus it may be better for your long term epistemic score that you do what is epistemically irrational.
Let r be any number strictly between 1/3 and 2/3 but other than 1/2. Suppose you live only for two days. A fair coin is tossed. After your first day, if your credence on the first day was r, the result of the coin toss is revealed to you. Otherwise, it is not revealed to you.
ReplyDeleteThe epistemically rational strategy is, surely, strategy A: assign credence 1/2 to heads during the first day, and then keep it at 1/2.
But now consider strategy B: assign credence r to heads during the first day, and then update according to what you learn.
I think I can prove:
Theorem: For EVERY strictly proper scoring rule S, the expected total S-score for B is better (i.e., lower) than the expected total S-score for for A.
(And of course the same is true for average S-scores, since on both scenarios you live for the same length of time.)
The proof looks like just some pretty easy fiddling with the representation of scoring rules in terms of strictly convex functions.
I think you have a typo in your second sentence.
ReplyDeleteRoughly, when p is true, and one assigns credence r to p, then a scoring rule measures the distance between r and 1, while when p is true, the scoring rule measures the distance between r and 0.
Should be 'while when p is false' ?
Hey Dr. Pruss!
ReplyDeleteI am considering this:
If I am in Las Vegas and someone has flipped a fair coin heads, nine times in a row. There is a special announcement made and one of the bystanders will be randomly chosen to make a prediction about the following toss. At no cost to this person they will win a million dollars if they choose correctly the next toss. They would be crazy not to pick heads?
Thanks, fixed!
ReplyDeleteAs for your example, you'd have good reason to be dubious about the assumption that the coin (and its flipping) was fair. But if you stick to being sure it's fair, then you need to stick to thinking heads and tails are equally likely.
A streak is a streak and we assume it will end. But when? Why bet against it?
ReplyDelete