First person survivalship bias?

Suppose I take a nasty fall while biking. But I remain conscious. Here is the obvious first thing for a formal epistemologist to do: increase my credence in the effectiveness of this brand of helmets. But by how much?

In an ordinary case of evidence gathering, I simply conditionalize on my evidence. But this is not an ordinary case, because if things had gone otherwise—namely, if I did not remain conscious—I wouldn’t be able to update or think in any way. It seems like I am now subject to a survivorship bias. What should I do about that? Should I simply dismiss the evidence entirely, and leave unchanged my credence in the effectiveness of helmets? No! For I cannot deny that I am still conscious—my credence for that is now forced to be one. If I leave all my other credences unchanged, my credences will become inconsistent, assuming they were consistent before, and so I have to do something to my other credences to maintain consistency.

It is tempting to think that perhaps I need to compensate for survivorship bias in some way, perhaps updating my credence in the effectiveness of the helmet to be bigger than my priors but smaller than the posteriors of a bystander who had the same priors as I did but got to observe my continued consciousness without a similar bias, since they would have been able to continue to think even were I to become unconscious.

But, no. What I should do is simply update on my consciousness (and on the impact, but if I am a perfect Bayesian agent, I have already done that as soon as it was evident that I would hit the ground), and not worry about the fact that if I weren’t conscious, I wouldn’t be around to update on it. In other words, there is no such problem as survivorship bias in the first person, or at least not in cases like this.

To see this, let’s generalize the case. We have a situation where the probability space is partitioned into outcomes E₁, ..., E_n, each with non-zero prior credence. I will call an outcome E_i normal if on that outcome you would know for sure that E_i has happened, you would have no memory loss, and would be able to maintain rationality. But some of the outcomes may be abnormal. I will have a bit more to say about the kinds of abnormality my argument can handle in a moment.

We can now approach the problem as follows: Prior to the experiment—i.e., prior to the potentially incapacitating observation—you decide rationally what kind of evidence update procedures to adopt. On the normal outcomes, you get to stick to these procedures. On the abnormal ones, you won’t be able to—you will lose rationality, and in particular your update will be statistically independent of the procedure you rationally adopted. This independence assumption is pretty restrictive, but it plausibly applies in the bike crash case. For in that case, if you become unconscious, your credences become fixed at the point of impact or become scrambled in some random way, and you have no evidence of any connection between the type of scrambling and the rational update procedure you adopted. My story can even handle cases where on some of the abnormal outcomes you don’t have any credences, say because your brain is completely wiped or you cease to exist, again assuming that this is independent of the update procedure you adopted for the normal outcomes.

It turns out to be a theorem that under conditions like this, given some additional technical assumptions, you maximize expected epistemic utility by conditionalizing when you can, i.e., whenever a normal outcome occurs. And epistemic utility arguments are formally interchangeable with pragmatic arguments (because rational decisions about wager adoption yield a proper epistemic utility), so we also get a pragmatic argument. The theorem will be given at the end of this post.

This result means we don’t have to worry in firing squad cases that you wouldn’t be there if you weren’t hit: you can just happily update your credences (say, regarding the number of empty guns, the accuracy of the shooters, etc.) on your not being hit. Similarly, you can update on your not getting Alzheimer’s (which is, e.g., evidence against your siblings getting it), on your not having fallen asleep yet (which may be evidence that a sleeping pill isn’t effective), etc., much as a third party who would have been able to observe you on both outcomes should. Whether this applies to cases where you wouldn’t have existed in the first place on one of the items in the partition—i.e., whether you can update on your existence, as in fine-tuning cases—is a more difficult question, but the result makes some progress towards a positive answer. (Of course, it woudn’t surprise me if all this were known. It’s more fun to prove things oneself than to search the literature.)

Here is the promised result.

Theorem. Assume a finite probability space. Let I be the set of i such that E_i is normal. Suppose that epistemic utility is measured by a proper accuracy scoring rule s_i when E_i happens for i ∈ I, so that the epistemic utility of a credence assignment c_i is s_i(c_i) on E_i. Suppose that epistemic utility is measured by a random variable U_i on E_i (not dependent on the choice of the c_j for j ∈ I) for i not in I. Let U(c)=∑_i∈I 1_Ei ⋅ s_i(c_i)+∑_i∉I 1_Ei ⋅ U_i. Assume you have consistent priors p that assign non-zero credence to each normal E_i, and the expectation E^p of the second sum with respect to these priors is well defined. Then the expected value of U(c) with respect to p is maximized when c_i(A)=p(A ∣ E_i) for i ∈ I. If additionally the scoring rules are strictly proper, and the p-expectation of the second sum is finite, then the expected value of U(c) is uniquely maximized by that choice of c_i.

This is one of those theorems that are shorter to prove than to state, because they are pretty obvious once fairly clearly formulated.

Normally, all the s_i will be the same. It's worth thinking if any useful generalization is gained by allowing them to be different. Perhaps there is. We could imagine situtions where depending on what happens to you, your epistemic priorities rightly change. Thus, if an accident leaves you with some medical condition, knowing more about that medical condition will be valuable, while if you don't get that medical condition, the value of knowing more about it will be low. Taking that into account with a single scoring rule is apt to make the scoring rule improper. But in the case where you are conditioning on that medical condition itself, the use of different but individually proper scoring rules when the condition eventuates and when it does not can model the situation rather nicely.

Proof of Theorem: Let c_i be the result of conditionalizing p on E_i. Then the expectation of s_i(c_i′) with respect to c_i is maximized when (and only when, if the conditions of the last sentence of the theorem hold) c_i′=c_i by propriety of s_i. But the expectation of s_i(c_i′) with respect to c_i equals 1/p(E_i) times the expectation of 1_Ei ⋅ s_i(c_i′) with respect to p. So the latter expectation is maximized when (and only when, given the additional conditions) c_i′=c_i.