Self-locating evidence and bearers of epistemic good

In the case of non-epistemic goods, it’s an obvious feature of life that someone there is a choice to be made by an individual between their own first-order good and the first-order good of the community—each requires the sacrifice of the other. In the case of epistemic goods, this is less obvious.

In the pragmatic case, the typical reason for such competition between goods is due to limited resources. This, of course, also happens in the epistemic sphere. Suppose Alice is much more intellectually talented than Bob, but only Bob has the money to go to university. If Bob spends the money on himself, he will gain private epistemic goods, but will contribute little epistemically to society as a whole. But if he gives the money to Alice, she may become a brilliant scholar or scientist, significantly contributing to society’s knowledge.

More interesting than these, however, are cases of competition between private and communal epistemic goods that are not due to epistemic resources. I find it interesting that some cases of self-locating evidence appear to be such.

Suppose there are ten billion people in the world, currently isolated from one another. A device produced by a mad scientist has a 99.9% chance at noon today of triggering a death ray that randomly kills 99.9999% of the world population. Noon has just passed. You are still alive. Should you think the device worked? Sleeping Beauty style arguments say “No”. This time I want to think about this in terms of individual epistemic goods. In N runs of the device, 0.001N runs will have you survive because the device doesn’t trigger and 0.999 ⋅ 0.000001N runs will have you survive despite the device triggering. Thus, the vast majority of the runs where you survive are runs where the device didn’t trigger. Hence, it’s best for you individually to adopt the epistemic policy of thinking the device didn’t trigger.

But on the other hand, suppose we all adopt the epistemic policy of thinking the device didn’t trigger. Then 99.9% of the time, we are unanimously collectively wrong. And if we all adopt the epistemic policy of thinking the device did trigger, then 99.9% of the time, we are unanimously collectively right. It seems thus that if we look at the epistemic goods of society, then a policy of thinking the device did trigger is best.

If this is right, it points to a potential diagnosis of why the problems about self-locating evidence (doomsday, multiverses, Sleeping Beauty, etc.) are so difficult. For there may be different bearers of epistemic goods at play—say, society vs. the individual—and it could be that different answers are appropriate depending on whose goods we are pursuing. Maybe.

Second person knowledge and reciprocal interaction

A number of philosophers have posited a special “second-person knowledge”, expressible by phrases like “Bob and Alice know each other”. It is often claims that second-person knowledge requires reciprocal interpersonal interaction.

I think this is false. Suppose Alice and Bob live on Earth but have not yet had any interaction. There is a Twin Earth, where by chance everything happens like on Earth, and there are Twin-Alice and Twin-Bob there. Suddenly Earth and Twin-Earth diverge, as follows.

• Twin-Alice and Twin-Bob are teleported away for a vacation on another planet. They don’t enter into our story after this.

• Bob is teleported to Twin-Earth, where he takes the place of Twin-Bob, without noticing anything being different.

• On Earth, Bob is replaced by Robo-Bob, who is a robot who looks just like Bob, but operates on non-intelligent software that generates random human-like behavior.

• On Twin-Earth, Twin-Alice is replaced by Robo-Alice, who looks just like Alice, but operates on non-intelligent software that generates random human-like behavior.

• On Earth, Alice meets Robo-Bob, and they have what to all appearances is a reciprocal interpersonal relationship of the sort that generates mutual second-person knowledge.

• On Twin-Earth, Bob meets Robo-Alice, and they have a similar “relationship”, with Bob behaving normally for his character.

• In these “relationships”, the behavior of Alice and Bob is such that it would be normal for them (given their respective characters) in the context of a normal interpersonal relationships.

At this point, Alice and Bob think that the above “relationships” yield second-person knowledge, but they don’t, because these relationships are with a randomly behaving non-intelligent robot. Let’s add this:

• By chance, on Earth, Robo-Bob happens to behave just like Bob behaves on Twin-Earth.

• By chance, on Twin-Earth, Robo-Alice happens to behave just like Alice behaves on Earth.

Still, Alice and Bob don’t have second-person knowledge in virtue of the “relationships” with robots. For one, they don’t even know with whom that relationship would be. But there is a way in which they have something like Gettiered second-person knowledge with an unknown person. Finally, one more thing happens, a big reveal.

• Alice and Bob are informed of everything that happened above, and they are given knowledge of who the real Alice and Bob are, say by being shown genuine properly labeled photographs of one another.

At this point, I think a case can be made that Alice’s knowledge of Bob is sufficiently close to the kind of knowledge that she would have had if she had been interacting with Bob rather than Robo-Bob. Alice might say: “Since I know that the real Bob was behaving to Robo-Alice just as Robo-Bob was behaving to me, while Robo-Alice was responding to Bob just as I was to Robo-Bob, and both persons were behaving in the normal way for their character, I know as much about Bob from Robo-Bob’s behavior as I would have had I had a normal relationship with Bob.” And Bob might say the same thing, mutatis mutandis.

But there was no reciprocity. Alice’s behavior does affect Bob once Bob learns that Alice behaved just like Robo-Alice, and Bob’s behavior does affect Alice once Alice learns about Bob’s actual behavior, but this is a pair of one-way interactions rather than reciprocal.

There are three things one might say here:

1. Alice and Bob have second-person knowledge of each other.

2. Alice and Bob don’t have second-person knowledge of each other, but what they have has all of the epistemic value of second-person knowledge.

3. There is something complex about interaction that I have a hard time putting my finger on that makes for a relevant difference.

Gettiered by degrees

Consider a standard Gettier case. A cutout of a sheep in a field hides a sheep behind it. At that distance, the cutout looks just like a sheep. You have a justified true belief that there is a sheep, but you don’t know it (or so the story goes).

Now imagine that cutout is to some degree transparent, so some of the whiteness you see is in fact from the sheep, and some from the cutout. Consider the continuum of cases as the cutout goes from fully opaque to full transparent. Perhaps it fades from opaque to transparent as you’re looking—all without you knowing that it is fading. When it’s fully or nearly opaque, you are Gettiered and don’t know there is a sheep. When it’s fully or nearly fully transparent, you know there is a sheep.

Supposing that knowledge has a distinctive value over and beyond the value of justified true belief, it seems plausible to think that this value increases monotonically with the transparency of the cutout. If the cutout is becoming more and more transparent before your eyes, you are gaining epistemic value, without noticing you are doing so.

It’s an interesting question: What kind of a function is there from cutout-transparency to value? Is it continuous, or is there a transparency threshold for knowledge at which it jumps discontinuously? If it is continuous, is it linear?

I have to confess that these kinds of questions seem a bit silly, and this gives some ammunitition to the thought that knowledge does not have a distinctive value.

Lifetime epistemic value

Suppose I discover some fact that I never end up using for anything, or even occurrently thinking about after the discovery. Now, knowledge is good. If I learn the fact earlier in life, then I will have had the knowledge for a longer period of time. So is it better for me to have learned the fact earlier in life?

I doubt it. Consider two scenarios. On the first, I learn what the capital of Zambia is just before I enter a ten-year coma. On the second, I learn it right right after I exit the coma. Learning it before the coma gives me ten more years of knowing it. But that seems a worthless gain. I conclude that in the case of non-occurrent knowing, it doesn’t matter much how long I know.

What about for occurrent knowledge? Other things being equal, if I learn some fact earlier in life, I will occurrently know the fact more times. Is that valuable?

I am less sure. But consider a daily ritual where every morning after waking up, before I am capable of any serious intellectual activity, I think to myself: Sheep have four legs. Thereby, I greatly increase the number of instances in which that piece of knowledge is being occurrently known. Again, this doesn’t seem to be worth the bother.

So it seems that neither for non-occurrent or occurrent knowledge is there non-instrumental value in knowing the thing for a longer period of time. Of course, there typically is instrumental value in knowing something for a longer period of time, both instrumental epistemic value—you can use it in your intellectual investigations of more things—and often instrumental pragmatic value.

This suggests the following. If an agent never loses knowledge, then the lifetime non-instrumental value of their knowledge depends on what they have come to know, not on when they have come to know it. The analogous thesis for perfect Bayesian agents and scoring rules is that their lifetime epistemic utility is the epistemic accuracy score at the latest point in their lives. (If we apply this to Sleeping Beauty, we are apt to get halving. But we shouldn’t apply this to Sleeping Beauty, as she forgets about her first wakeup.)

Things are more complicated in the case of agents who do lose knowledge, whether to memory loss, irrationality or misleading evidence. If we count such an agent’s lifetime non-instrumental epistemic value based on all that they have ever known, that means that if they lost knowledge of p, there is no gain to them from getting it back. But obviously they are better off epistemically if they do get it back. Things get messy and complicated now. A short-period loss in old age doesn’t seem as bad as a case where you found out something early in life and then didn’t have it for the rest of your life.

This is getting messy.

The epistemic value of experiments

You perform an experiment and are going to rationally update on its results. It seems that you should expect this to be good for your epistemic utility as compared to non-performance of the experiment.

Not always! Silly case: Your boss has tasked you with performing a boring chemistry experiment. If you do the experiment, you will find out very little. But if you don’t do it, you will find out a lot about the range of swear words that your boss knows.

What makes this case silly is that you should really think of it as a choice of which experiment to perform, one in chemistry or one in psychology, and in this case the psychology experiment is the more interesting one.

So if we want to say that an experiment can be expected to improve your epistemic utility, we need to be a bit more careful. We need to ensure that non-performance of the experiment doesn’t itself generate information.

But it always does. At the very least, non-performance of the experiment generates the information that the experiment has not been performed by you. You find out something about yourself, and that might far outweigh the value of anything you find out from the experiment. Granted, you also find out something about yourself by performance of the experiment, but it is easy to imagine cases where what you find out by non-performance is more significant. For instance, it could be that your refusal to perform the experiment shows that you have a very specific and rare personality type, while your performance of the experiment gives you nothing so specific.

Suppose, for instance, that you score your epistemic utility by bits of information. The experiment consists in bending down to see which side an unusual coin lying on the ground is facing—that’s one bit of information. Your prior probability that you will look at the coin is 3/4: you are the sort of person who tends to look. So by looking at the coin, you will gain 1 − log₂(3/4) = 1.4 bits, mostly regarding the coin but also a little bit about yourself. By not looking at the coin, you will gain 0 − log₂(1/4) = 4 bits, all about yourself. Better not to look!

Of course, there are Newcomb-like issues here.

Lesson: The principle that performing a non-trivial experiment should be expected to improve epistemic utility is going to be difficult to formulate.

Predictability and epistemic utility

You’re thinking whether to become an assembly-line worker or an artist. Then you reflect on the value of knowledge. And you become a factory worker, on the grounds that if you become an assembly-line worker, you will know what you’ll be doing every working day of your future, but if you’re an artist, your activities will be unpredictable.

Some remarks. First, there is something perverse about using the value of knowledge in this way. The normal way to pursue the value of knowledge is to find out things that are independent of your pursuit. But here you are pursuing knowledge by making there be less to know about the world (or your world). Yet, paradoxically, it sure seems like the line of thought above makes sense.

Second, the my initial story depends on Molinism being false. For if there are comprehensive subjective conditionals of free will, then by becoming an artist you get to know the conditionals about what you would do in the various artistic situations you’re in. But on the assembly line story, you don’t get to know these. So the Molinist doesn’t have the paradox. I suppose that’s a bit of evidence for Molinism.

Life-time epistemic utility

I’ve been thinking about the diachronic aspects of epistemic utility. In the case of non-epistemic utility, we can get a decent first approximation to life-time utility by adding up (or, if time is continuous, integrating) momentary utility. But I think this works less well for the epistemic case. For many things of purely epistemic importance, figuring them out is much more important than when one figures them out. (Granted, figuring them out earlier is instrumentally epistemically valuable, because it gives one more time to use leverage the knowledge to figure other things.)

Here’s an extreme version of encoding the value of “figuring something out”. Assuming one does not suffer from mental decline, the epistemic value of one’s life is the epistemic value of the very last moment of it. It’s interesting to note that this won’t work. For imagine that no matter what other credences you had at a given time, you always set the credence of “This is the last moment of my life” to one, while being careful to (inconsistently) make no use of this credence in update. If only the last moment counts, this modification to your credences would be a good idea: it makes sure that when the last moment comes, you get the epistemic utility credit for it.

I suspect that other weightings that favor later over earlier beliefs will suffer from a similar problem—they make it a good idea to err on the side of pessimism about how close death is.

But at the same time, I think some sort of favoring of later beliefs over earlier ones seems appropriate. I don’t know how to resolve this difficulty.

Epistemic utilities and death

In the previous post, I proved that we get a proper scoring rule if we compute epistemic utilities as follows. We start with our current credence assignment, consider what credence assignment we will have in the future after we update on some further evidence, and then score that. I then suggested that one could get a lifetime epistemic utility by adding up the epistemic utilities over all the moments of life, and as long as death wasn’t random—as long as the lifespan was fixed—this would generate a proper scoring rule. I then said that if death is random (as it is) it might be the case that you don’t get a proper scoring rule.

My conjecture was wrong. You still get a proper scoring rule despite random death. It’s easy to see this. The basic idea is this. Suppose that you might die the next moment. This partitions the probability space into two subsets, D and L, for death and life. Your current credence is p. Next moment, on D, your credence either doesn’t exist (because you don’t exist or you exist in some supernatural state where you don’t have credences) or doesn’t count (because I am after lifetime credences). Thus, the appropriate way to do a forward-looking scoring of your credence p is to score it s(p_L) on L and 0 on D, where p_L is the result of conditionalizing your credence on the evidence L (after all, if you are alive, you will conditionalize on being alive), and s is some proper scoring rule. In other words, your forward looking score is s_L(p) = 1_L ⋅ s(p_L).

Is this score proper? Yes! For by propriety of s we have:

• E_pL(s(p_L)) ≥ E_pL(s(q_L)).

But this is the same as:

• (p(L))⁻¹E_p(1_L⋅s(p_L)) ≥ (p(L))⁻¹E_p(1_L⋅s(q_L)).

Multiplying both sides by p(L) (I am assuming a non-zero probability of survival), we get:

• E_p(s_L(p)) ≥ E_p(s_L(q)).

We can combine this with a more complex set of future investigations as in the previous post, and things will still work.

It is crucial to the above argument that when you’re alive, you can tell you’re alive. I suppose that’s not always true. When you’re asleep, you are alive, but can’t tell you’re alive. So to generalize beyond the above toy example, replace death with unconsciousness or something like that.

More on strong open-mindedness

For the last couple of days I have been exploring what I like to call strongly open-minded accuracy scoring rules. It’s well known that every proper scoring rule is open-minded in the sense that it never requires you to reject free information: the expected epistemic utility of updating on the free information is always at least as good as your current expected epistemic utility. It’s strictly open-minded provided that in non-trivial cases (i.e., when the information has a non-zero probability of having statistical relevance to the credences you are scoring) you are required to accept the free information.

Now there are two reasons why one might accept free information about some proposition q. First, you might be wrong about q: your credence may be high while q is false or your credence might be low while q is true. Second, even if you are right about q, the free information may boost your credence in the right direction. I say that a scoring rule is strongly open-minded provided that it licenses you to accept and update on the free information even if you disregard the first consideration. We can then tack on “strictly” if it requires you to do so in non-trivial cases. In the case of a strongly open-minded scoring rule, your acceptance of free information is not a sign of doubt in your propositions—it is not a way of hedging your bets—and thus is arguably compatible with faith in the propositions being evaluated.

A strongly open-minded scoring rule can also be characterized in the following way. There is a more ordinary kind of epistemic paternalism where I might have reason to block another from receiving free information on the grounds that this information could mislead due to the fact that the other has different likelihoods from the ones I think are right. For instance, if too many people have an unjustified mistrust of Dr. Smith such that they are likely to believe the opposite of what Dr. Smith’s experiments reveal, there is reason to give a grant to someone else, because Dr. Smith’s experiments are likely to lead people away from the truth, for no fault of Dr. Smith’s. Call this likelihood-based paternalism. But there is another kind of motivation of the refusal of free information for another, which we might call pure-risk-based paternalism. Even if someone else has the same likelihoods as you do—trusts Dr. Smith just as you do—perhaps the risk that Dr. Smith’s experiments will, by pure chance, provide evidence away from the truth is enough to justify not funding these experiments.

I’ve been collecting results about these issues. Here’s what I seem t have so far, though I have to emphasize that sometimes the proofs are just in my head and I might be wrong. I will specialize on scoring rules for a single proposition, given as a pair of functions T and F, where T(x) is the value of having credence x when the proposition is true and F(x) is the value of having credence x when the proposition is false.

1. A scoring rule sometimes calls for pure-risk-based paternalism if and only if it is not strongly open-minded.

2. A scoring rule that’s strongly open-minded is open-minded.

3. A scoring rule (T,F) is (strictly) strongly open-minded if and only if xT(x) and (1−x)F(1−x) are both (strictly) convex.

4. The logarithmic scoring rule is strictly strongly open-minded. The Brier and spherical rules are not strongly open-minded.

5. If a proper scoring rule is generated by the Schervisch-style integral representation T(x) = T(1/2) + ∫_x^1/2(1−t)b(t)dt and F(x) = F(1/2) + ∫_1/2^xtb(t)dt and b is sufficiently differentiable, then the scoring rule is strongly open-minded if and only if the derivative of log b(x) lies between (3x−2)/[x(1−x)] and (3x−1)/[x(1−x)].

6. A strongly open-minded scoring rule whose logarithm is sufficiently differentiable is unbounded.

7. [Item deleted as I discovered it to be false.]

8. For any credences p and r such that 1/2 < r and p < r, there is a strictly proper scoring rule and a situation where the scoring rule calls for the individual with credence r to have purely-risk-based epistemic paternalism for that hypothesis.

Lying and epistemic utility

Epistemic utility is the value of one’s beliefs or credences matching the truth.

Suppose your and my credences differ. Then I am going to think that my credences better match the truth. This is automatic if I am measuring epistemic utilities using a proper scoring rule. But that means that benevolence with respect to epistemic utilities gives me a reason to shift your credences to be closer to mine.

At this point, there are honest and dishonest ways to proceed. The honest way is to share all my relevant evidence with you. Suppose I have done that. And you’ve reciprocated. And we still differ in credences. If we’re rational Bayesian agents, that’s presumably due to a difference in prior probabilities. What can I do, then, if the honest ways are exhausted?

I can lie! Suppose your credence that there was once life on Mars is 0.4 and mine is 0.5. So I tell you that I read that a recent experiment provided a little bit of evidence in favor of there once having been life on Mars, even though I read no such thing. That boosts your credence that there was once life on Mars. (Granted, it also boosts your credence in the falsehood that there was such a recent experiment. But, plausibly, getting right whether there was once life on Mars gets much more weight in a reasonable person’s epistemic utilities than getting right what recent experiments have found.)

We often think of lying as an offense against truth. But in these kinds of cases, the lies are aimed precisely at moving the other towards truth. And they’re still wrong.

Thus, it seems that striving to maximize others’ epistemic utility is the wrong way to think of our shared epistemic life.

Maximizing others’ epistemic utility seems to lead to a really bad picture of our shared epistemic life. Should we, then, think of striving to maximize our own epistemic utility as the right approach to one’s individual epistemic life? Perhaps. For maybe what is apt to go wrong in maximizing others’ epistemic utility is paternalism, and paternalism is rarely a problem in one’s own case.

Why do we like being confident?

We like being more confident. We enjoy having credences closer to 0 or 1. Even if the proposition we are confident in is one that is such that it is a bad thing that it is true, the confidence itself, abstracted from the badness of the state of affairs reported by the proposition, is something we enjoy.

Here is a potential justification of this attitude in many cases. We can think of the epistemic utility of one’s credence r in a proposition p as measured by an accuracy scoring rule given by two functions T(r) and F(r), where T(r) gives the value of having credence r in p when p is actually true and F(r) gives the value when p is actually false. Most people thinking about scoring rules think they should satisfy the technical condition of being strictly proper. But strict propriety implies that the function V(r) = rT(r) + (1−r)F(r) is strictly convex. Now suppose the scoring rule is also symmetric, so that T(r) = F(1−r). Then V(r) is a strictly convex function that is symmetric about r = 1/2. Such a function has its minimum at r = 1/2, and is strictly decreasing on [0,1/2] and strictly increasing on [1/2,1]. But the function V(r) measures your expectation of your epistemic utility. How happy you are about your credence, perhaps, corresponds to your expectation of your epistemic utility. So you are most unhappy at credence 1/2, and you get happier that closer you are to 0 or 1.

OK, it’s surely not that?!

Combining epistemic utilities

Suppose that the right way to combine epistemic utilities or scores across individuals is averaging, and I am an epistemic act expected-utility utilitarian—I act for the sake of expected overall epistemic utility. Now suppose I am considering two different hypotheses:

• Many: There are many epistemic agents (e.g., because I live in a multiverse).

• Few: There are few epistemic agents (e.g., because I live in a relatively small universe).

If Many is true, given averaging my credence makes very little difference to overall epistemic utility. On Few, my credence makes much more of a difference to overall epistemic utility. So I should have a high credence for Few. For while a high credence for Few will have an unfortunate impact on overall epistemic utility if Many is true, because the impact of my credence on overall epistemic utility will be small on Many, I can largely ignore the Many hypothesis.

In other words, given epistemic act utilitarianism and averaging as a way of combining epistemic utilities, we get a strong epistemic preference for hypotheses with fewer agents. (One can make this precise with strictly proper scoring rules.) This is weird, and does not match any of the standard methods (self-sampling, self-indication, etc.) for accounting for self-locating evidence.

(I should note that I once thought I had a serious objection to the above argument, but I can't remember what it was.)

Here’s another argument against averaging epistemic utilities. It is a live hypothesis that there are infinitely many people. But on averaging, my epistemic utility makes no difference to overall epistemic utility. So I might as well believe anything on that hypothesis.

One might toy with another option. Instead of averaging epistemic utilities, we could average credences across agents, and then calculate the overall epistemic utility by applying a proper scoring rule to the average credence. This has a different problematic result. Given that there are at least billions of agents, for any of the standard scoring rules, as long as the average credence of agents other than you is neither very near zero nor very near one, your own credence’s contribution to overall score will be approximately linear. But it’s not hard to see that then to maximize expected overall epistemic utility, you will typically make your credence extreme, which isn’t right.

If not averaging, then what? Summing is the main alternative.

More on averaging to combine epistemic utilities

Suppose that the right way to combine epistemic utilities across people is averaging: the overall epistemic utility of the human race is the average of the individual epistemic utilities. Suppose, further, that each individual epistemic utility is strictly proper, and you’re a “humanitarian” agent who wants to optimize overall epistemic utility.

Suppose you’re now thinking about two hypotheses about how many people exist: the two possible numbers are m and n, which are not equal. All things considered, you have credence 0 < p₀ < 1 in the hypothesis H_m that there are m people and 1 − p₀ in the hypothesis H_n that there are n people. You now want to optimize overall epistemic utility. On an averaging view, if H_m is true, if your credence is p₁, your contribution to overall epistemic utility will be:

• (1/m)T(p₁)

and if H_m is false, your contribution will be:

• (1/n)F(p₁),

where your strictly proper scoring rule is given by T, P. Since your credence is p₁, by your lights the expected value after your changing your credence to p₀ will be:

• p₀(1/m)T(p₁) + (1−p₀)(1/n)F(p₁) + Q(p₀)

where Q(p₀) is the contribution of other people’s credences, which I assume you do not affect with your choice of p₁. If m ≠ n and T, F is strictly proper, the expected value will be maximized at

• p₁ = (p₀/m)/(p₀/m+(1−p₀)/n) = np₀/(np₀+m(1−p₀)).

If m > n, then p₁ < p₀ and if m < n, then p₁ > p₀. In other words, as long as n ≠ m, if you’re an epistemic humanitarian aiming to improve overall epistemic utility, any credence strictly between 0 and 1 will be unstable: you will need to change it. And indeed your credence will converge to 0 if m > n and to 1 if m < n. This is absurd.

I conclude that we shouldn’t combine epistemic utilities across people by averaging the utilities.

Idea: What about combining them by computing the epistemic utilities of the average credences, and then applying a strictly proper scoring rule, in effect imagining that humanity is one big committee and that a committee’s credence is the average of the individual credences?

This is even worse, because it leads to problems even without considering hypotheses on which the number of people varies. Suppose that you’ve just counted some large number nobody cares about, such as the number of cars crossing some intersection in New York City during a specific day. The number you got is even, but because the number is big, you might well have made a mistake, and so your credence that the number is even is still fairly low, say 0.7. The billions of other people on earth all have credence 0.5, and because nobody cares about your count, you won’t be able to inform them of your “study”, and their credences won’t change.

If combined epistemic utility is given by applying a proper scoring rule to the average credence, then by your lights the expected value of the combined epistemic utility will increase the bigger you can budge the average credence, as long as you don’t get it above your credence. Since you can really only affect your own credence, as an epistemic humanitarian your best bet is to set your credence to 1, thereby increasing overall human credence from 0.5 to around 0.5000000001, and making a tiny improvement in the expected value of the combined epistemic utility of humankind. In doing so, you sacrifice your own epistemic good for the epistemic good of the whole. This is absurd!

I think the idea of averaging to produce overall epistemic utilities is just wrong.

Adding or averaging epistemic utilities?

Suppose for simplicity that everyone is a good Bayesian and has the same priors for a hypothesis H, and also the same epistemic interests with respect to H. I now observe some evidence E relevant to H. My credence now diverges from everyone else’s, because I have new evidence. Suppose I could share this evidence with everyone. It seems obvious that if epistemic considerations are the only ones, I should share the evidence. (If the priors are not equal, then considerations in my previous post might lead me to withhold information, if I am willing to embrace epistemic paternalism.)

Besides the obvious value of revealing the truth, here are two ways to reason for this highly intuitive conclusion.

First, good Bayesians will always expect to benefit from more evidence. If my place and that of some other agent, say Alice, were switched, I’d want the information regarding E to be released. So by the Golden Rule, I should release the information.

Second, good Bayesians’ epistemic utilities are measured by a strictly proper scoring rule. But if Alice’s epistemic utilities for H are measured by a strictly proper (accuracy) scoring rule s that assigns an epistemic utility s(p,t) to a credence p when the actual truth value of H is t, which can be zero or one. By definition of strict propriety, the expectation by my lights of what Alice’s epistemic utility for a given credence should be is strictly maximized when that credence equals my credence. Since Alice shares the priors I had before I observed E, if I can make E evident to her, her new posteriors will match my current ones, and so revealing E to her will maximize my expectation of her epistemic utility.

So far so good. But now suppose that the hypothesis H = H_N is that there exist N people other than me, and my priors assign probability 1/2 to there being N and 1/2 to its being n, where N is much larger than n. Suppose further that my evidence E ends up significantly supporting hypothesis H_n, so that my posterior p in H_N is smaller than 1/2.

Now, my expectation of the total epistemic utility of other people if I reveal E is:

• U_R = pNs(p,1) + (1−p)ns(p,0).

And if I conceal E, my expectation is:

• U_C = pNs(1/2,1) + (1−p)ns(1/2,0).

If we had N = n, then it would be guaranteed by strict propriety that U_R > U_C, and so I should reveal. But we have N > n. Moreover, s(1/2,1) > s(p,1): if some hypothesis is true, a strictly proper accuracy scoring rule increases strictly monotonically with the credence. If N/n is sufficiently large, the first terms of U_R and U_C will dominate, and hence we will have U_C > U_R, and thus I should conceal.

The intuition behind this technical argument is this. If I reveal the evidence, I decrease people’s credence in H_N. If it turns out that the number of people other than me actually is N, I have done a lot of harm, because I have decreased the credence of a very large number N of people. Since N is much larger than n, this consideration trumps considerations of what happens if the number of people is n.

I take it that this is the wrong conclusion. On epistemic grounds, if everyone’s priors are equal, we should release evidence. (See my previous post for what happens if priors are not equal.)

So what should we do? Well, one option is to opt for averaging rather than summing of epistemic utilities. But the problem reappears. For suppose that I can only communicate with members of my own local community, and we as a community have equal credence 1/2 for the hypothesis H_n that our local community of n people contains all agents, and credence 1/2 for the hypothesis H_n + N that there is also a number N of agents outside our community much greater than n. Suppose, further, that my priors are such that I am certain that all the agents outside our community know the truth about these hypotheses. I receive a piece of evidence E disfavoring H_n and leading to credence p < 1/2. Since my revelation of E only affects the members of my own commmunity, depending on which hypothesis is true, if p is my credence after updating on E, the relevant part of the expected contribution to the utility of revealing E with regard to hypothesis H_n is:

• U_R = p((n−1)/n)s(p,1) + (1−p)((n−1)/(n+N))s(p,0).

And if I conceal E, my expectation contribution is:

• U_C = p((n−1)/n)s(1/2,1) + (1−p)((n−1)/(n+N))s(p,0).

If N is sufficiently large, again U_C will beat U_R.

I take it that there is something wrong with epistemic utilitarianism.

Bayesianism and epistemic paternalism

Suppose that your priors for some hypothesis H are 3/4 while my priors for it are 1/2. I now find some piece of evidence E for H which raises my credence in H to 3/4 and would raise yours above 3/4. If my concern is for your epistemic good, should I reveal this evidence E?

Here is an interesting reason for a negative answer. For any strictly proper (accuracy) scoring rule, my expected value for the score of a credence is uniquely maximized when the credence is 3/4. I assume your epistemic utility is governed by a strictly proper scoring rule. So the expected epistemic utility, by my lights, of your credence is maximized when your credence is 3/4. But if I reveal E to you, your credence will go above 3/4. So I shouldn’t reveal it.

This is epistemic paternalism. So, it seems, expected epistemic utility maximization (which I take it has to employ a strictly proper scoring rule) forces one to adopt epistemic paternalism. This is not a happy conclusion for expected epistemic utility maximization.

A feature of the logarithmic scoring rule

Accuracy scoring rules measure the epistemic utility of having some credence assignment. For simplicity, let’s assume that all credence assignments are probabilistically coherent. A strictly proper scoring rule has the property that always by one’s own lights, the expected value of one’s actual credence assignment is better than that of any other credence assignment.

A well-known fact is that a strictly proper scoring rules always makes it rational to update on non-trivial evidence. I.e., by one’s present lights, the expected epistemic utility after examining and updating on non-trivial evidence will be higher than the expected epistemic utility of ignoring that evidence. We might put this by saying that a strictly proper scoring rule is strictly open-minded.

The logarithmic scoring rule makes the score of assigning credence r be log r when the hypothesis is true and log (1−r) when the hypothesis is false. It is strictly proper and hence strictly open-minded.

The logarithmic scoring rule, however, satisfies a condition even stronger than strict open-mindedness. This condition is easiest to describe in a binary case where one is simply evaluating the score of one’s credence in a single hypothesis H. Assuming some non-triviality assumptions, it turns out that not only is the expected epistemic utility increased by examining evidence, but the expected epistemic utility conditional on H is increased by examining evidence. (This is a pretty easy calculation.)

So what?

Well, there are several reasons this matters. First, on my recent account of what it is to have a no-hedge commitment to a hypothesis H, if your epistemic utilities are measured by some scoring rules (e.g., Brier) and you have a no-hedge commitment to H but you do not have credence 1 in H, then you will sometimes have reason to refuse to look at evidence. But the above fact about the logarithmic scoring rule shows that this is not so for the logarithmic scoring rule. With the logarithmic scoring rule, it makes sense to look at the evidence even if you have a no-hedge commitment to H—i.e., even if all your betting behavior is “as if H”.

Second, let’s imagine that I run a funding agency and you come to me with an interest in doing some experiment relevant to a hypothesis H. Let’s suppose that the relevant epistemic community agrees on the relevant likelihoods with respect to the evidence obtainable from the experiment, and is perfectly rational, but differs with regard to the priors of H. I might then have this paternalistic worry about funding the experiment. Even though updating on the results of the experiment by my lights is expected to benefit me epistemically, if a strictly proper scoring rule is the appropriate measure of benefit, it may not be true that by my lights other members of the community will benefit epistemically from updating on the results of the experiment. I may, for instance, be close to certain of H, and think that some members of the community have credences that are sufficiently high that the benefit to them of getting a boost in credence in H from the experiment is outweighed by the risk of misleading evidence. If it is my job to watch out for the epistemic good of the community, this could give me reason to refuse funding.

But not so if I think the logarithmic rule is the right way to evaluate epistemic utility. If everyone shares likelihoods, and we differ only in priors for H, and everyone is rational, then when we measure epistemic utility with the logarithmic rule, I have a positive expectation of the epistemic utility effect of examining the experiment’s results on each member of the community. This is easily shown to follow from my above observation about the logarithmic scoring rule. (By my lights the expectation of a fellow community member’s epistemic utility after updating on the experimental results is a weighted sum of an expectation given H and an expectation given not-H. Each improves given the experiment.)

Discontinuous epistemic utilities

I used to take it for granted that it’s reasonable to make epistemic utilities be continuous functions of credences. But this is not so clear to me right now. Consider a proposition really central to a person’s worldview, such as:

• life has (or does not have) a meaning

• God does (or does not) exist

• we live (or do not live) in a simulation

• morality is (or is not) objective.

I think a case can be made that if a proposition like that is in fact true, then there is a discontinuous upward jump in epistemic utility as one goes from assigning a credence less than 1/2 to assigning a credence more than 1/2.

When truth makes you do less well

One might think that being closer to the truth is guaranteed to get one to make better decisions. Not so. Say that a probability assignment p₂ is at least as true as a probability assignment p₁ at a world or situation ω provided that for every event E holding at ω we have p₂(E)≥p₁(E) and for every event E not holding at ω we have p₂(E)≤p₁(E). And say that p₂ is truer than p₁ provided that strict inequality holds in at least one case.

Suppose that a secret integer has been picked among 1, 2 and 3, and p₁ assigns the respective probabilities 0.5, 0.3, 0.2 to the three possibilities while p₂ assigns them 0.7, 0.1, 0.2. Then if the true situation is 1, it is easy to check that p₂ is truer than p₁. But now suppose that you are offered a choice between the following games:

• W₁: on 1 win $2, on 2 win $1100, and on 3 win $1000.

• W₂: on 1 win $1, on 2 win $1000, and on 3 win $1100

If you are going by p₁, you will choose W₁ and if you are going by p₂, you will choose W₂. But if the true number is 1, you would be better off picking W₁ (getting $2 instead of $1), so the truer probabilities will lead to a worse payoff. C’est la vie.

Say that a scoring rule for probabilities is truth-directed if it never assigns a poorer score for a truer set of probabilities. The above example shows that a proper scoring rule need not be truth-directed. For let s(p)(n) be the payoff you will get if the secret number is n and you make your decision between W₁ and W₂ rationally on the basis of probability assignment p (with ties broken in favor of W₁, say). Then s is a proper (accuracy) scoring rule but the above considerations show that s(p₂)(1)<s(p₁)(1), even though p₂ is truer at 1. In fact, we can get a strictly proper scoring rule that isn’t truth-directed if we want: just add a tiny multiple of a Brier accuracy score to s.

Intuitively we would want our scoring rules to be both proper and truth-directed. But given that sometimes we are pragmatically better off for having less true probabilities, it is not clear that scoring rules should be truth-directed. I find myself of divided mind in this regard.

How common is this phenomenon? Roughly it happens whenever the truer and less-true probabilities disagree on ratios of probabilities of non-actual events.

Proposition: Suppose two probability assignments are such that there are events E₁ and E₂ with probabilities strictly between 0 and 1, with ω₁ in neither event, and such that the ratio p₁(E₁)/p₁(E₂) is different from the ratio p₂(E₁)/p₂(E₂). Then there are wagers W₁ and W₂ such that p₁ prefers W₁ and p₂ prefers W₂, but W₁ pays better than W₂ at ω₁.

Truth directed scoring rules on an infinite space

A credence assignment c on a space Ω of situations is a function from the powerset of Ω to [0, 1], with c(E) representing one’s degree of belief in E ⊆ Ω.

An accuracy scoring rule s assigns to a credence assignment c on a space Ω and situation ω the epistemic utility s(c)(ω) of having credence assignment c when in truth we are in ω. Epistemic utilities are extended real numbers.

The scoring rule is strictly truth directed provided that if credence assignment c₂ is strictly truer than c₁ at ω, then s(c₂)(ω)>s(c₁)(ω). We say that c₂ is strictly truer than c₁ if and only if for every event E that happens at ω, c₂(E)≥c₁(E) and for every event E that does not happen at ω, c₂(E)≤c₁(E), and in at least one case there is strict inequality.

A credence assignment c is extreme provided that c(E) is 0 or 1 for every E.

Proposition. If the probability space Ω is infinite, then there is no strictly truth directed scoring rule defined for all credences, or even for all extreme credences.

In fact, there is not even a scoring rule that strictly truth directed when restricted to extreme credences, where an extreme credence is one that assigns 0 or 1 to every event.

This proposition uses the following result that my colleague Daniel Herden essentially gave me a proof of:

Lemma. If PX is the power set of X, then there is no function f : PX → X such that f(A)≠f(B) whenever A ⊂ B.

Now, we prove the Proposition. Fix ω ∈ Ω. Let s be a strictly truth directed scoring rule defined for all extreme credences. For any subset A of PΩ, define c_A to be the extreme credence function that is correct at ω at all and only the events in A, i.e., c_A(E)=1 if and only if ω ∈ E and E ∈ A or ω ∉ E and E ∉ A, and otherwise c_A(E)=0. Note that c_B is strictly truer than c_A if and only if A ⊂ B. For any subset A of PΩ, let f(A)=s(c_A)(ω).

Then f(A)<f(B) whenever A ⊂ B. Hence f is a strictly monotonic function from PPΩ to the reals. Now, if Ω is infinite, then the reals can be embedded in PΩ (by the axiom of countable choice, Ω contains a countably infinite subset, and hence PΩ has cardinality at least that of the continuum). Hence we have a function like the one the Lemma denies the existence of, a contradiction.

Note: This suggests that if we want strict truth directedness of a scoring rule, the scoring rule had better take values in a set whose cardinality is greater than that of the continuum, e.g., the hyperreals.

Proof of Lemma (essentially due to Daniel Herden): Suppose we have f as in the statement of the Lemma. Let ON be the class of ordinals. Define a function F : ON → A by transfinite induction:

• F(0)=f(⌀)

• F(α)=f({F(β):β < α}) whenever α is a successor or limit ordinal.

I claim that this function is one-to-one.

Let H_α = {F(δ):δ < α}.

Suppose F is one-to-one on β for all β < α. If α is a limit ordinal, then it follows that F is one-to-one on α. Suppose instead that α is a successor of β. I claim that F is one-to-one on α, too. The only possible failure of injectivity on α could be if F(β)=F(γ) for some γ < β. Now, F(β)=f(H_β) and F(γ)=f(H_γ). Note that H_γ ⊂ H_β since F is one-to-one on β. Hence f(H_β)≠f(H_γ) by the assumption of the Lemma. So, F is one-to-one on ON by transfinite induction.

But of course we can’t embed ON in a set (Burali-Forti).

Misleadingness simpliciter

It is quite routine that learning a truth leads to rationally believing new falsehoods. For we all rationally believe many falsehoods. Suppose I rationally believe a falsehood p and I don’t believe a truth q. Then, presumably, I don’t believe the conjunction of p and q. But suppose I learn q. Then, typically, I will rationally come to believe the conjunction of p and q, a falsehood I did not previously believe.

Thus there is a trivial sense in which every truth I learn is misleading. But a definition of misleadingness on which every truth is misleading doesn’t seem right. Or at least it’s not right to say that every truth is misleading simpliciter. What could misleadingness simpliciter be?

In a pair of papers (see references here) Lewis and Fallis argue that we should assign epistemic utilities to our credences in such a way that conditioning on the truth should never be bad for us epistemically speaking—that it should not decrease our actual epistemic utility.

I think this is an implausible constraint. Suppose a highly beneficial medication has been taken by a billion people. I randomly sample a hundred thousand of these people and see what happened to them in the week after receiving the medication. Now, out of a billion people, we can expect about two hundred thousand to die in any given week. Suppose that my random sampling is really, really unlucky, and I find that fifty thousand of the people in my sample died a week because of the medication. Completely coincidentally, of course, since as I said the medication is highly beneficial.

Based on my data, I rationally come to believe the importantly false claim that the medication is very harmful. I also come to believe the true claim that half of my random sample died a week after taking the medication. But while that claim is true, it is quite unimportant except as misleading evidence for the harmfulness of the medication. It is intuitively very plausible that after learning the truth about half of the people in my sample dying, I am worse off epistemically.

It seems clear that in the medication case, my data is true and misleading in a non-trivial way. This suggests a definition of misleadingness simpliciter:

• A proposition p is misleading simpliciter if and only if one’s overall epistemic utility goes down when one updates on p.

And this account of misleadingness is non-trivial. If we measure epistemic utility using strictly proper scoring rules, and if our credences are consistent, then the expected epistemic value of updating on the outcome of a non-trivial observation is positive. So we should not expect the typical truth to be misleading in the above sense. But some are misleading.

From this point of view, Lewis and Fallis are making a serious mistake: they are trying to measure epistemic utilities in such a way as to rule out the possibility of misleading truths.

By the way, I think I can prove that for any measure of epistemic utility obtained by summing a single strictly proper score across all events, there will be a possibility of misleadingness simpliciter.

Final note: We don’t need to buy into the formal mechanism of epistemic utilities to go with the above definition. We could just say that something is misleading iff coming to believe it would rationally make one worse off epistemically.