Thursday, April 14, 2022

Some possible progress on continuous scoring rules and dominance in an infinite case

On finite sample spaces, we have the Pettigrew-Nielsen-Pruss domination theorem for strictly proper scoring rules that are continuous when restricted to the probabilities that shows that the score of any non-probability is dominated by the score of a probability. Last year, I showed that for a reasonable sense of “continuous”, this is not true on countably infinite sample spaces (when we take probabilities to be countably additive; for if we take probabilities to be finitely additive, there are no strictly proper scoring rules).

In the comments, Ian then suggested that we want our scoring rule to be continuous on all credences, not just the probabilities.

Here are two preliminary responses, though not all the details of the proof of the second have yet been checked, so I could just be wrong.

First, what happens seems to depend on the topology on the space of credences. Credences can be thought of as functions from PΩ to [0,1]. One possibility is to take the space of credences to get the product topology on [0,1]PΩ. In that case, there is no continuous strictly proper (or even quasi-strictly proper) scoring rule. This follows from the uncountability of PΩ which shows that any countable intersection of neighborhoods of a probability function will contain infinitely many non-probability functions, so that any continuous score will have the property that for every probability there is a non-probability that gets the same score.

But, second, another reasonable topology on [0,1]PΩ is the ℓ(PΩ) topology. This topology is easily seen to be equivalent on the probabilities to the 1(Ω) topology (where a probability p on PΩ corresponds to a function p* ∈ ℓ1(Ω) defined by p*(x) = p({x})). The example in my earlier post was a score s that was equal to the spherical score on all the probabilities and s(c)(n) = 1/2(n+1) for any non-probability credence, where we identify Ω with the natural numbers.

Let Q be the space of probability functions on PΩ. Let d(c) = infp ∈ Qc − p be the distance from c to Q. We can prove that d(c) = 0 iff c is a probability, and d is continuous in our topology. Let ϕ(c) = 0 if d(c) ≥ 1/4 and ϕ(c) = 4d(c) if d(c) < 1/4. This will be a continuous function. Now define s(c)(n) = ϕ(c)/2(n+1) + (1−ϕ(c))c*(n)/∥c*2, where c*(n) = c({n}), and where the second summand is deemed to be zero if ϕ(c) = 1 (regardless of the denominator). I haven’t checked all the details yet, but this s looks continuous to me in the relevant norm, but the domination result is false for any non-probability c. The important point is that the function c ↦ ∥c*2 is continuous and non-zero for c such that d(c) < 1/4, and that’s one of the points I might yet have an error in.


Unknown said...

Pettigrew-Nielsen-Pruss theory governs the principles of final article inclusion, and if the possibilities are limited then the probability ratio is much better. Over the past year, I have noticed that it does not fit in with many archetypes in the “ongoing” thought model (a commonly accepted opportunity).

"The rules of our unions should apply not only to our disputes, but to all disputes," Janan said.

There are two main answers, but not all the details from the second source are verified, so I could be wrong.

What happens first depends on the nature of the witness paper. Certificate [0,1] can be considered a software application. [0,1] software may replace authentication of a product. However, the principle of continuous growth is wrong (or almost entirely). The software comes from endless workplaces, where there is unlimited power at the juncture of potential work so that uninterrupted space is not possible.

James Reilly said...

Hi Professor Pruss. A commentor on this blog recently gave an argument against omniscience based on the halting problem. It goes (more or less) as follows: suppose that God has complete knowledge about any program P either halting H or not halting ~H for an input I. Since that knowledge is complete, we might put it into a list like so:

P1: {H for input I1, H for input I2, ~H for input I3...}
P2: {~H for input I1, H for input I2, ~H for input I3...}
P3: {H for input I1, ~H for input I2, ~H for input I3...}

And so on. Now, we can construct a program P' in the following way, which is not on the list of God's knowledge: P' is H for an Ik, if and only if Pk is ~H for that Ik.

Hence, P': {~H for input I1, ~H for input I2, H for input I3...}, which is differing from P1 in its first input, P2 in its second input, and so on ad infinitum. This is intended to show that God's knowledge itself is incomplete and incompletable. I was wondering if you had any thoughts on this particular argument. Thanks!

Gordon said...

James Reilly,

I am going to take a stab at addressing your question even though you addressed it to Professor Pruss - I hope I am not violating the etiquette of the blog by doing so...

It seems to me that P' describes a function but not a program. The function described by P' is uncomputable, since to compute it you would need to be able to compute for an arbitrary program and an arbitrary input whether the program halts. Therefore, there is no program P'.

So, it does not make any sense to ask whether God can answer whether program P' returns H or ~H for any particular input, because there is no program P'. However, God clearly can evaluate the function described by P' for any input - since by your own assumption God knows P1, P2, ..., then He knows P'(n) - it is just ~Pn(n). The fact that function P’ does not show up in your list P1, P2, … is of no concern since P1, P2, … is a complete list of programs, but P’ is not a program. So, I don’t think that you can show that God’s knowledge is limited in this manner.

Zsolt Nagy said...

Really, James? You couldn't give a proper credit to me for that argument of mine with simply putting my name ", Zsolt Nagy," behind those words "A commentator on this blog"? Come on.
Besides that, here is my inspiration for that argument of mine:
"Impossible Programs (The Halting Problem)" by Undefined Behavior
In my opinion the "oracle" sounds very much similar to an "omniscient god" especially in their functionalities.
So I guess, that that person, "Undefined Behavior", or even Alan Turing also have deserved some credit for that specific and particular argument against the existence of an "omniscient god".

Apropos, functionality!
Gordon, have you ever heard of "functional programming"?
I guess, that you have never heard of that.
And yes, it should be a great concern to you, that the function/instruction/program P' is not showing up in that supposedly complete knowledge of God about functions/instructions/programs P either halting H or not halting ~H given for a specific, particular or any input I, since function/instruction/program P' is actually a program, which is not contained in God's supposedly complete knowledge about such functions/instructions/programs P halting H or not halting ~H for a given input I.

PS: To administrator.
These "off-topic" comments are already staying up here not for any time, but for quite some time. (- Do you see, what I did here?)
Hm. Is there a reason for that?
Maybe the Easter holidays?
I guess so.

James Reilly said...


I wasn't entirely sure whether your account was still allowed on the blog, and I also didn't want to poison the well by bringing you into it (considering your history of juvenile behavior), I figured a comment containing your name was likely to go unaddressed.

As for the argument itself, it seems that Gordon is obviously correct: if God knows whether H or ~H for any input I(k) into any program P(k), then he automatically also knows P'. This is because P' is defined in terms of the list of programs P, which (as we've assumed by hypothesis) God already knows. Thus, if God wanted to evaluate the function P', he could simply look at the relevant program and input, and know that the answer is the inversion of that.

ZsoltnagY said...


If a comment/argument containing my name (or for this matter of fact any name) wouldn't be likely to go unaddressed, only because of that containment of that name rather than because of the made comment/argument itself, then that would be rather a fallacious act of ad hominem.
Arguing from "poisoning the well" is a special case of that kind of fallacy. I never get those kinds of points and justifications.
It just appears to me, that a particular issue is being tried to be avoided with that reference to "poisoning the well"/"the well being poisoned" rather than that particular issue is being tried to be properly addressed and solved.
I'm only trying to be honest and yes, at same time I demand and request the opposition to be also that honest.
If doing so is supposed to be a "crime", then yes, I'm guilty of doing that "crime".
Sure, one might accuse me of being "guilty" and "poisoning the well". But than how is that in any given way addressing anything relevant to the made argument or solve any issue or problem at hand? I don't get it and I specially don’t get, how requesting of “being honest” or “some honesty” can lead to such absurd punishments especially made by Theists or Christians...

ZsoltnagY said...

... As for the argument itself, - as for your made argument and point here:
First, your point is not even remotely similar yet alone being the same as Godon's point:

Gordon's point: "Zsolt's argument against the existence of an omniscient god is false, because the supposed 'program' P' is not actually a program. P' is a function. So the fact that function P’ does not show up in God's supposedly complete knowledge P1, P2, … is of no concern since P1, P2, … is a complete knowledge of programs from God, but P’ is not a program."
My response: Even if P' is a function. It is still considerable and actually to be considered here as a program. So the fact that function/program P’ does not show up in God's supposedly complete knowledge P1, P2, … is of great concern since P1, P2, … is supposedly a complete knowledge of programs from God, but apparently not containing that function/program P', but which is supposed to be the case.

Your point here James: "Zsolt's argument against the existence of an omniscient god is false, because the supposed program P' is actually contained in that supposedly complete knowledge of God about any program P either halting H or not halting ~H for a given input I."
My response: If so - if program P' is actually contained in that supposedly complete knowledge of God about any program P either halting H or not halting ~H for a given input I, then I can just "cook up"/define another program P'', which is apparently not contained in that supposedly complete knowledge of God about any program P either halting H or not halting ~H for a given input I:

Program P'' is H for input Ik, if and only if program Pk is ~H for input Ik
for any k=1,2,3,...
AND Program P'' is H for input I', if and only if program P' is ~H for input I'.
With this definition we can foretell any result of either H or ~H for any input I:
program P'': {"~H for input I1","H for input I2","~H for input I3",...}∪{"H, if and only if ~H for input I'"}

So this new program P'' is yet again not to be found on that supposedly complete knowledge of God about any program P either halting H or not halting ~H for a given input I, since the foretolled results of program P'' differs at least in one entry/result for each and any program on that supposedly complete knowledge of God about the halting problem for any program.
Besides that, program P'' apparently will "produce"/entail a logical contradiction "H, if and only if ~H" for input I'. So also according to this the supposedly complete knowledge of God about the halting problem of programs is actually incomplete and also not really completable.

Wesley C. said...

@James & @Zsolt,

This whole argument assumes that God's knowledge is structured like a list, and so non-listable truths can't be known because they can't be listed. The problem with this is that there's no reason to assume God's knowledge is like a list - you could just as easily say that God's omniscience is such that all things which are true or real or intelligible are known by God in a sense, and that's a categorisation so simple and universal that no counterexample can be thought of without also denying that the truth is real. Just as we can say all truths are, well, true - or all programs are real. It's an inescapable triviality.

ZsoltnagY said...

Really, Wesley? There's no reason to assume God's knowledge is like a list?
If that knowledge is to be described by logic and mathematics - by any kind of a formal language, then that knowledge described by logic and mathematics is listable - then there is a bijection to a subset of natural numbers or the whole set of natural numbers itself - Gödel numbering.
Besides that, if "mathematics is the language of nature" or given "The Unreasonable Effectiveness of Mathematics in the Natural Sciences" and if mathematics is a listable formal language, then I guess, that there are good reasons, very good reasons to assume God's knowledge is like a list or is even listable.

Sure, you might rather want to restrict or simplify God's knowledge in such a way, that "no counterexample can be thought of without also denying that the truth is real", whatever that is supposed to mean. Sure, then "all truths are true" and "all programs are programs".
But then by what means is supposed to be the case, that "all programs are real"?
Sure, some programs are real. But all? How?
That's as bold of an assertion as the assertion of the PSR.

Wesley C. said...

@Zsolt Nagy

Really? Are these your arguments? First of all, God's knowledge is beyond math and logic, including formal systems, given His very omniscience in knowing all things, including those not describable by any formal system. Second, just because nature may have a mathematical structure and God created it that way in no way implies God's knowledge is limited by math or logic in the way you want to imply. God knows math and logic but also more than those.

Third, when I say all programs are real I mean all the possible ideas and possible programs are real - insofar as they are things that are knowable or exist at least as non-realised possibilities.

Dominik Kowalski said...

A truth of a particular is necessarily known, since all that exists depends on being. In that sense every true proposition can be listed as facts known by being (God). Then there are knowable, possible truths, like the Twin Towers collapsed on 9/11, which are possibly true and are actually in some worlds. We could also formulate propositions which do the same work, which however aren't specified to any time, e.g. "There are 8 billion people on earth."

Now we have two options, neither I regard as problematic.

1) We can follow Barry Miller in his analysis of being and individuals. Here, due to the priority of essence to the particular esse of each individual (priority of nature of individual being, though not, of course, over being as such), then the individual prior to its existence is unknowable, the individual just is the specific set of properties it possesses brought into being, however, obviously, before that act of coming about, there's no individual to be known. There are thus new truths, which, in some sense, could be said to be learned about. But this language is misleading. To learn is to signal an act of understanding of something already there. But there was no truth to be learned, it was a new truth *brought about*. And this truth, going full circle back to the dependence on being additionally to the afromentioned origin within it. So we have a peculiar additional category of truth, one which if it isn't known, that's due to it not existing at all, not even in the simpliciter sense. The charge of Being thus not being omniscient (if we concede for the sake of illustration the Aristotelian notion of the Absolute), would amount to nothing more than a category error.

Also, I don't think it needs to be mentioned though, this of course isn't restricted to humans or even biological entities, but to every being, since every being is an individual.

2) The second way is the Steinian one, endorsed by a friend of mine. I'm unsure of it and lean towards 1), but we can add it into the extreme case I want to use here: Suppose every individual is a priori knowable, be it because individuality is a limitation on being, so that God knows uncreated individuals by knowing himself. Suppose additionally an Avicennan ontology in which every essence posseses the most minimal version of being, so that every possible being is a being merely waiting to be actualized, be in a third realm or as divine ideas. And suppose further Lewisian modal realism, so that every possible world is actualized. I'm actually very sympathetic to the latter due to the explanatory power, and it's also the best ontology available to the naturalist.

Now we can put aside the example of the computer program (the argument should have never been formulated in the way it is, it's form is unnecessarily complicated, while mine is easier to comprehend), every possible version of it is actual in the one gigantic block created, something intriguingly endorsed and developed by Almeida. We now have everything and everything is actual. So, which truth is now supposedly unknowable?

Wesley C. said...

@Dominik, What ZN means by listable or lists is related to mathematical concepts of listability in the sense of finding a one-to-one bijection of things - basically, if you had an infinite line of socks and an infinite line of hats, you could give each individual sock a hat to pair it with. Or that you could have an infinite list with each individual of the infinity listed on the list.

Now some classes of things like the Real Numbers are so large they defy listability, meaning there is always a member that won't be on even an infinite linear list. See Cantor's Diagonal theorem for more, and Set Theory in general as well.

The problem with ZN's objection is he assumes God's knowledge just is structured like a linear infinity such as the Naturals, and there's not only no good reason for this assumption it's also contradicted by the fact omniscience clearly includes all things, and so God is in a sense united to all things in a supra-categorical or non-linear way.

As for theories of being - I don't disagree with anything, but I just thought it may be interesting to add (for the sake of curiosity) that one of the differences between Scotism and Thomism is that Scotism rejects the De Ente et Essentia arguments for the real distinction between essence and existence (in the sense of separability, so that essence and existence are parts, whereas for Scotism they're not metaphysical parts), as well as the idea that God is Existence Itself, and prefer to speak of existential metaphysics in terms of intrinsic modes of being (ways of being real in a sense) rather than existence and essence being parts - the way they describe the divine essence is such that it can only "exist" under the modes of infinity and intrinsic necessity.

Basically, there is a diversity of approaches even in classical metaphysics, and that's interesting in itself.

As for the Steinian view, that sounds kinda similar to the Scotist view of divine ideas as being intrinsically individual, not just universal - haecceity being the individuating factor. Also not sure if one needs to view essences and forms as being just limitations of being, as essences and forms themselves are a real thing, and so participate in the real in some sense.

Zsolt Nagy said...

@Wesley, you shouldn't use words in a manner, which are not meant to be used
Otherwise you might mislead others and even yourself with such incorrect usages of words.
You have obviously a very different understanding about the word "linearity".
The real number line is "linear" and square numbers are not "linear" and yet there is a one-to-one correspondence between those "non-linear" square numbers and the "linear" natural numbers.
So then by what means is there no one-to-one correspondence between the supposedly "non-linear" knowledge of God and the "linear" natural numbers?
Because the one is supposedly"non-linear" and the other one is "linear"?
If so, then you clearly don't properly understand, what "linearity" supposed to mean and that "linearity" has not much to do with there being or there not being such a correspondence.
So yes, "linearity" as properly understood is a non sequitur in regards to there supposedly being no correspondence between God's knowledge and the natural numbers.

Besides that, God's knowledge being supposedly "supra-categorical"/"uncountably infinite" might explain, why there is no one-to-one correspondence between God's knowledge and the natural numbers.
So omniscience clearly includes all things and is without any limitations.
If so, then such an omniscient being shouldn't have any problem to build a machine/program solving any halting problem.
Yet apparently the existence of such a machine/program leads to a contradiction:
"Turing & The Halting Problem - Computerphile"
"Given the omnipotence of God could God create such a heavy rock, which God can not move?"
I guess, that at least "logical limitations" to omniscience should be considered as such "logical limitations" to omnipotence have also already been considered given otherwise the possibility and plausibility to such contradictions rising from that omniscience and omnipotence.

Dominik Kowalski said...

No attribute defies logic though, there are no real contradictions in existence, as such objects would just amount to nothing. So if such machines amount to contradictions I see no reason to affirm that it should be in principle knowable how to build them

Alexander R Pruss said...

The move from what God knows to what he can design machines to do is problematic as Dominik points out.

To use an example from Nicholas Rescher, there logically cannot be a machine that can correctly answer any possible yes-or-no question. For we could then ask the machine: "Are you going to answer my question in the negative?" And if there logically cannot be such a machine, then we can't expect God to be able to create such a machine, just as we can't expect God to be able to create a square circle or to truthfully say that 2+2=5.

ZsoltnagY said...

I don't think, that if there logically cannot be such a yes-or-no answering machine, then God is only not to be expected to be able to create such a machine, but also if there logically cannot be such a yes-or-no answering machine, then also God himself is not to be expected to be able to provide answers for any yes-or-no questions, since otherwise the same contradiction could and can be expected.
Then why even bother postulating/expecting God to be such an omniscient being, if God is not even to be expected to be able to provide answers for any yes-or-no questions - only for "some" yes-or-no questions?
That's what I'm not getting at.

Alexander R Pruss said...

The function c ↦ ∥c*∥_2 doesn't look continuous to me now.

Alexander R Pruss said...

But it's not hard to get the continuity in the l^infty(P Omega) norm by using: Dugundji's extension theorem: