Are there multiple models of the naturals that are "on par"?

Assuming the Peano Axioms of arithmetic are consistent, we know that there are infinitely many sets that satisfy them. Which of these infinitely many sets is the set of natural numbers?

A plausible tempting answer is: “It doesn’t matter—any one of them will do.”

But that’s not right. For the infinitely many sets each of which is a model of the Peano Axioms are not isomorphic. They disagree with each other on arithmetical questions. (Famously, one of the models “claims” that the Peano Axioms are consistent and another “claims” that they are inconsistent, where we know from Goedel that consistency is equivalent to an arithmetical question.)

So it seems that with regard to the Peano Axioms, the models are all on par, and yet they disagree.

Here’s a point, however, that is known to specialists, but not widely recognized (e.g., I only recognized the point recently). When one says that some set M is a model of the Peano Axioms, one isn’t saying quite as much as the non-expert might think. Admittedly, one is saying that for every Peano Axiom A, A is true according to M (i.e., M⊨A). But one is not saying that according to M all the Peano Axioms are true. One must be careful with quantifiers. The statement:

1. For every Peano Axiom A, according to M, A is true.

is different from:

2. According to M, all the Peano Axioms are true.

The main technical reason there is such a difference is that (2) is actually nonsense, because the truth predicate in (2) is ineliminable and cannot be defined in M, while the truth predicate in (1) is eliminable; we are just saying that for any Peano Axiom A, M⊨A.

There is an important philosophical issue here. The Peano Axiomatization includes the Axiom Schema of Induction, which schema has infinitely many formulas as instances. Whether a given sequence of symbols is an instance of the Axiom Schema of Induction is a syntactic matter that can be defined arithmetically in terms of the Goedel encoding of the sequence. Thus, it makes sense to say that some sequence of symbols is a Peano Axiom according to a model M, i.e., that according to M, its Goedel number satisfies a certain arithmetical formula, I(x).

Now, non-standard models of the naturals—i.e., models other than our “normal” model—will contain infinite naturals. Some of these infinite naturals will intuitively correspond, via Goedel encoding, to infinite strings of symbols. In fact, given a non-standard model M of the naturals, there will be infinite strings of symbols that according to M are Peano Axioms—i.e., there will be an infinite string s of symbols such that its Goedel number g_s is such that I(g_s). But then we have no way to make sense of the statement: “s is true according to M” or M⊨s. For truth-in-a-model is defined only for finite strings of symbols.

Thus, there is an intuitive difference between the standard model of the naturals and non-standard models:

1. The standard model N is such that all the numbers that according to N satisfy I(x) correspond to formulas that are true in N.

2. A non-standard model M is not such that all the numbers that according to M satisfy I(x) correspond to formulas that are true in M.

The reason for this difference is that the notion of “true in M” is only defined for finite formulas, where “finite” is understood according to the standard model.

I do not know how exactly to rescue the idea of many inequivalent models of arithmetic that are all on par.