Tuesday, September 11, 2012

Defending infinitary frequentism from some arguments

Frequentism defines probabilities in terms of long-term frequencies of outcomes. This doesn't work very well with finite frequencies—for one, it's going to conflict with physics in that finite sequence frequentism can only yield rational numbers as probabilities while quantum physics is quite happy to yield irrational numbers. As a result frequentism is often extended to suppose a hypothetical infinite sequence of data for defining frequencies. Alan Hajek has a paper that gives fifteen arguments against such a frequentism. Fourteen of them are strong arguments against standard hypothetical frequentism (I am unmoved by argument 15 involving infinitesimals, since I doubt that infinitesimal probabilities are much use to us).

But it turns out that one can formulate a frequentism that escapes or partly escapes some of Hajek's arguments.

A representative of these six arguments is the observation going back to De Finetti that probabilities defined via frequencies fail to satisfy the Kolmogorov Axioms (arguments 13 and 14). But my modified frequentist probabilities satisfy the Kolmogorov Axioms.

For simplicity, our data will be real valued, but the extension to Rn is easy. Let s=(sn) be our sequence of real numbers in R. For any subset A of R, let Fn(s,A) be the proportion of s1,...,sn that is in A. Let L(s,A) be the limit of Fn(s,A) if that limit exists, and otherwise L(s,A) is undefined.

Say that a (classical) probability measure m on the Borel subsets of R fits s provided that for all subintervals I of R, L(s,I) is defined and L(s,I)=m(I).

If there is a probability measure m that fits s, let a frequentist probability measure Ps defined by s be the completion of m (basically, the completion of a measure sets the measure of all subsets of null sets to be measurable and have measure zero).

Proposition:

  1. If s defines a frequentist probability measure, it defines a unique frequentist probability measure.
  2. Suppose that P is a probability measure and X=(Xn) is a sequence of independent identically distributed random variables. Let P1 be the measure on R defined by P1(A)=P(X1 in A). Then with probability one, X defines a frequentist probability measure on R which coincides with P1.

Because Ps is an ordinary Kolmogorovian probability measure, Hajek's arguments 13 and 14 do not apply. Argument 15 is anyway not very convincing, but is weakened since our version of frequentism handles the case of a dart thrown at [0,1] about as well as one can expect classical probabilities to handle it. (There is also a tension between arguments 13-14 and 15, in that probabilities involving infinitesimals are unlikely to be Kolmogorovian.) It is plausible that our frequentist probability measure will provide frequencies only when there is a probability, which makes argument 8 not apply, and non-uniqueness worries from argument 4 are ruled out by (1). I think the frequentist can bite the bullet on arguments 5 and 6, whether with standard frequentism or our modified version, given that the problem occurs only with probability zero.

Remark: The big philosophical problem with this is the reliance on intervals.

Quick sketch of proof:

Claim (1) is easy, because two Borel measures that agree on all intervals agree everywhere.

Claim (2) is proved by letting S be the collection of (a) all intervals with rational numbers as endpoints and (b) all singletons with non-zero P1 measure, and using the Strong Law of Large Numbers to see that for each member A of S almost surely L(X,A) exists and equals P1(A). But since S has countably members (obvious in the case of the intervals, but also easy in the case of the singletons), almost surely for every A in S we have L(X,A) existing. Moreover, almost surely, no singleton with null P1 measure will be hit by infinitely many of the Xn, and hence L(X,A) will be defined and equal to zero for all such singletons.

Thus there is a set W of P-measure one such that on W, we have L(x,A) existing and equal to P1(A) for every A that is either an interval with rational number endpoints or a singleton. Approximating any other interval A from below and above with monotone sequences of rational-number-ended intervals plus or minus one or two singletons, we can show that L(x,A) exists and equals P1(A) for any other interval everywhere on W.

6 comments:

  1. A perhaps somewhat neater definition is to define P_s as the completion of the Caratheodory extension of the pre-measure defined by L(s,-) on the semi-ring of intervals, when L(s,-) is in fact a pre-measure on the semi-ring of intervals.

    ReplyDelete
  2. Actually, while the main construction works fine for R^n, proving statement (2) in the case of R^n is going to be rather messier. But still, I think, doable.

    ReplyDelete
  3. Looks like the proof of (2) follows from the fact that n-dimensional rectangles are a Vapnik-Chervonenkis (VC) class, and a uniform strong LLN holds for such. See here and the references here.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete