Monday, August 30, 2021

Absence of evidence

It seems that the aphorism “Absence of evidence is not evidence of absence” is typically false.

For if H is a hypothesis and E is the claim that there is evidence for H, then E raises the probability of H: P(H|E)>P(H). But then (as long as P(E)>0, as Bayesian regularity will insist), it mathematically follows that P(∼H|∼E)>P(∼H). Thus the absence of evidence is evidence for the falsity (“absence”) of the hypothesis.

I think there is only one place where one can challenge this argument, namely the claim:

  1. If there is evidence for H, then the fact that there is evidence for H is itself evidence for H.

First, let’s figure out what (1) is saying. I think the best reading is that it presupposes some kind of notion of a body of first-order evidence—maybe all the stuff that human beings have ever observed—and says that if the actual contents of that body of first-order evidence supports H, then the fact that that body supports H itself supports H.

Here is a way to make this precise. We suppose there is some random variable O whose value (not real valued, of course) is all first-order observations humans ever made. Let W be the set of all possible values that O could take on. For simplicity, we can take W to be finite: there is a maximum number of observations a human can make in a lifespan, a finite resolution to each observation, and a maximum number of human beings who could have lived on earth. Let o0 be the actual value that O has. Let WH = {o ∈ W : P(H|O = o)>P(H)}.

Assuming we have Bayesian regularity, we can suppose O = o has non-zero probability for each o ∈ WH. Then the claim that there is evidence for H is itself evidence for H comes to this:

  1. P(H|O ∈ WH)>P(H).

And it is easy to check that this follows by finite conglomerability from the fact that P(H|O = o)>P(H) for each o ∈ WH.

There might be cases where we expect infinite conglomerability to be lacking. In those cases (1) would be dubious. Here is one such case. Suppose Alice and Bob each get a ticket from a fair infinite jar with tickets numbered 1,2,3,…. Alice looks at her ticket. Bob doesn’t look at his yet, but knows that Alice has looked at hers. Bob notes that whatever number Alice has seen, it is nearly certain that his number is bigger (there are infinitely many numbers bigger than Alice’s number and only finitely many smaller ones). Thus, Bob knows that the evidence available to humans supports the thesis that his number is bigger than Alice’s. But Bob’s knowing this is not actually evidence that his number is bigger than Alice’s, for until Bob actually observes one or the other number, he is in the same evidential position as before Alice looked at her ticket—and at that point, it is obvious that it’s not more likely that Bob’s ticket has a bigger number than Alice’s.

But apart from weird cases where conglomerability fails, (1) is true, and so absence of evidence is evidence of absence, assuming we have enough Bayesian regularity.

Perhaps a charitable reading of the aphorism that absence of evidence isn’t evidence of absence is just that absence of evidence isn’t always significant evidence of absence. That seems generally correct.


Andrew Dabrowski said...

Clever, but I think the phrase "absence of evidence" in ordinary usage corresponds not to -E, the negation of E, but rather to the indeterminacy of E: E has not been established as either true or false.

Alexander R Pruss said...

I don't think so. If that were true, the maxim would be too much a triviality. For if "there is an absence of evidence with respect to H" is a claim symmetric between H and not-H, then it's trivial that if there is absence of evidence with respect to H, there is no evidence for not-H. I take it, rather, the maxim is claiming that absence of evidence *for* H does not constitute (is not identical with) the presence of evidence against H.

Andrew Dabrowski said...

Well, trivial maxims are the most common kind...

And taking "absence of E" to mean -E is going too far in the other direction: it implies experiments experiments which might have supported H have failed, in which case I don't think anyone would argue that the probability of H hasn't gone down a little.

I've usually seen the phrase "absence of evidence is not evidence of absence" used by paranormal researchers. In that domain there are endless arguments about whether experiments/observations support psi/UFOs/etc., with no definitive results.