This is one of those annoying loosey-goosey big-picture posts.
Consider the Newtonian law of gravitation: F=Gmm'/r2. What should be the prior probability of that law?
Humean line of thought: Zero. After all, consider the continuum of laws of nature of the form F=Gmm'/rp, where p is some real number. The case where p=2 is just one case out of a continuum. And of course the schema F=Gmm'/rp is just one schema out of a continuum of schemata (consider, for instance, replacing the multiplication operations on the right hand side with a continuum of other operations). So the prior probability of F=Gmm'/r2 is simply zero.
Complexity line of thought: Moderate. After all, the elegant formula "F=Gmm'/r2" is by far simpler than the vast majority of its alternatives (most laws of the form F=Gmm'/rp have no finite expression, since in most cases the number p will be a real number with no finite mathematical description).
If the Humean line of thought is right, Bayesianism has no hope as a model of how scientific reasoning works. The Complexity line of thought allows for a Bayesian picture of scientific reasoning.
The Humean and Complexity lines of thought come with different pictures of how probabilities are to be assigned to situations. The Humean picture is based on the idea that you've got a bunch of fundamental physical entities, say particles, and then you generate situations by assigning them random fundamental physical properties. From that point of view, clearly the Newtonian law of gravitation has probability zero.
The Complexity line of thought presupposes a different picture. The picture is that probabilities are tied to linguistic expressions. That to generate the probability of a situation, you generate a random complete linguistic description of a world, and identify the probability of a situation with the probability that such a random description entails the situation.
But what a strange thing the Complexity line of thought is! It is as if our picture of the world was that the really central thing about the world wasn't the physical stuff and its fundamental physical properties, but the descriptions. It is as if reality were fundamentally linguistic, or at least explained by something linguistic, as if the cosmos came from a being who said: "Let it be the case that s", and we then assigned probabilities to different values of "s".
In other words, the Complexity line of thought is at heart not naturalistic. But of the two lines of thought, it is the one that is needed for a Bayesian picture of scientific reasoning.
The Complexity line of thought has technical problems, too. Suppose I perform some experiment and the result can be any real number between 0 and 1. The Complexity line of thought will, I think, assign probability one to the hypothesis that the result of the experiment is a finitely describable real number. But surely other real numbers are possible. So what is to be done? It is, I think, to move from the Complexity line of thought to a theistic line of thought focused on value (and a certain autonomy in nature can then a value, and that could allow for randomness and hence for indescribable real number outcomes).