Monday, October 19, 2015

A puzzle about testimony

You weigh a bag of marbles on a scale that you have no information about the accuracy of, and the scale says that the bag weighs 342 grams. If you have no background information about the bag of marbles, your best estimate of the weight of the bag is 342 grams. It would be confused to say: "I should discount for the unreliability of the scale and take my best estimate to be 300 grams." For if one has no information about the scale's accuracy, one should not assume that the scale is more likely to overestimate than to underestimate by a given amount. So far so good. Now, suppose that instead of your using the scale, you give me the bag, I hold it in my hand, and say: "That feels like 340 grams." Again, your best estimate of the weight will now be 340 grams. You don't know whether I am apt to overestimate or underestimate, so it's reasonable to just go with what I said.

But now consider a different case. You have no background information about my epistemic reliability and you have no evidence regarding a proposition p, but I inform you that I have some relevant evidence and I estimate the weight of that evidence at 0.8. It seems that the same argument as before should make you estimate the weight of the evidence available to me at 0.8. But that's all the evidence available right now to either of us, so you should thus assign a credence of 0.8 to p. But the puzzle is that this is surely much too trusting. Given no information about my reliability, you would surely discount, maybe assigning a credence of 0.55 (but probably not much less). Yet, doesn't the previous argument go through? I could be overestimating the weight of the evidence. But I could also be underestimating it. By discounting the probability, you are overestimating the probability of the denial of p, and that's bad.

There is, however, a difference between the weight of evidence and the weight of marbles. The weight of marbles can be any positive real number. And if we take really seriously the claim that there is no background information about the marbles, it could be a negative number as well. So we can reasonably say that I or the scale could equally be mistaken in the upward or the downward direction. However, if we know anything about probabilities, we know that they range between 0 and 1. So my estimate of 0.8 has more possibilities of being an overestimate than of being an underestimate. It could, for instance, be too high by 0.3, with the correct estimate of the weight of my evidence being 0.5, but it couldn't be too low by 0.3 for then the correct estimate would be 1.1. We can, thus, block the puzzling argument for trust. Though that doesn't mean the conclusion of the argument is wrong.

4 comments:

entirelyuseless said...

I don't see any real puzzle here. In real life you do have some background information about weights, and you do have some background information about probabilities. If the scale says that the bag of marbles weighs 342 pounds (instead of grams), I will certainly assume that the scale is wrong. And if someone tells me that an unidentified proposition has a 99% chance of being true, I will assume that it has about a 75% of being true. If he says 80%, I will say about 60%. That is because of my background knowledge, just as in the case of the scale.

Alexander R Pruss said...

I share something like your intuitions if they say the chance is 0.99. But I don't share your intuitions if they say that the chance is 0.80. In the latter case, I don't see what justifies thinking that they were rather more likely to overestimate than to underestimate the weight of their evidence.

Heath White said...

There is something odd here which I cannot quite put my finger on. When you say “I think the bag is 340 grams” and I take this as my best estimate, what I really have is a range of credences for propositions of the form “The bag weighs _____ grams.” I may think 340 is the most likely value but I will have pretty similar credences for 339 and 341. The farther away we get from 340, the lower my credences will be. Presumably the whole graph of my credences will approximate a bell curve. To say that my best estimate is 340 grams is to say the curve peaks at 340. But it does not say much about the breadth of the curve.

Suppose you tell me that you think the bag is 340 grams, and you are extremely confident of this, which is to say that your curve of credences peaks at 340 but is also very tall and skinny. If I adopt your estimate as my best estimate, it seems right that my curve ought to peak at the same place. On the other hand I don’t feel obliged to take on board your confidence, i.e. the shape of your curve. Mine might be much broader and fatter than yours, and rationally so.

It’s not obvious why this is rational. But maybe this: you are a bag-weighing instrument, of unknown accuracy. Still, probably not zero accuracy. So your deliverances about the weight of the bag are evidence, but not very good evidence. And while the peak of my curve ought to follow the evidence, the shape of my curve should follow the quality of the evidence.

Now suppose Zelda comes along and I ask her, “How good evidence is Alex’s testimony that the bag weighs 340 grams?” High numbers in answers to this question will have the tendency to give my curve of credences about the bag weight a sharp skinny peak, while low numbers will have the tendency to broaden and flatten it. She says, “Probability 0.8.” Now, it is not really true that this is my only evidence about the quality of your evidence. I have lived a few decades, heard many dubious confident assertions, and have some idea of who to trust. So perhaps Zelda’s opinion will make my curve of credences peak a little more sharply but not a lot.

Consider a different case, where I really do have no information about the quality of the evidence other than testimony. Zelda says, “I am thinking of a proposition which I will call the Zelda Conjecture. I have some evidence for this proposition. That evidence puts its probability at 0.8.” Since I do not know what the Zelda Conjecture is, nor what the evidence is, I think my best rational estimate is that the probability of the Zelda Conjecture is 0.8. But my curve of credences for THAT proposition would be pretty broad and flat.

I am not sure that answers all the questions. I am sure that it is a pretty unsystematic answer.

Alexander R Pruss said...

Heath:

Maybe this is due to my very particular idea of "best estimate" as the mathematical expectation of the correct value. The best estimate can be an estimate that for sure isn't true. Let's say you know that there is an even number of marbles in a bag, and you count twice. The first time you count 1006 marbles and the second time you count 1008 marbles. Your best estimate is 1007, even though you know for sure that that's not the right answer.

As applied to credences, here's how I'm thinking about the "best estimate". I am entering a lottery. I don't know how many tickets there are in the lottery, but there are only four possibilities: 100, 1000, 10000 and 100000. What should my credence be as to my winning (there is only one winner)? Well, it's (0.001 + 0.0001 + 0.00001 + 0.000001) / 4 which is about 0.00028. This is the case even though I know that this isn't actually my chance of winning on any of the four hypotheses. Nonetheless, 0.00028 is my "best estimate".