Could everything in existence be explained by an infinite causal regress? I've argued in print that the answer is negative. Here is another thought in favor of the negative answer.
Suppose we have a world where there is an infinite regress of past states and that there is some numerical time-varying property Q(t) in the universe (say, entropy or temperature or charge or volume) that keeps on changing in such a way that the property's having one the value it does in one year determines the value it has in another. Now, consider the claim that the limit of Q(t) as t goes to minus infinity is, say, q0. (Of course, Q(t) might not in fact converge to anything, but since we're stipulating, we can stipulate that it does.) If the regress explains everything, it explains why it was that the limit is q0. But even if a regress explains something, it surely doesn't explain the boundary values at the beginning of the regress, and q0 is precisely such a boundary value. It would be almost as absurd to say that the limit of the temperature in a room tended to 11.0 degrees as t approached noon from above because it was 11.1 at 12:10, and 11.06 at 12:05, and 11.05 at 12:05, and 11.01 at 12:01, and so on, as it would be to say that the temperature of the room was 11.0 degrees at noon because it had such-and-such values after noon.
Suppose that we still insist that the boundary value q0 is explained by the regress. Now suppose that we find another explanation, say in terms of an atemporal being creating the universe. What should we say? That we have two explanations of the same phenomenon, each on its own sufficient? That can happen, but surely that's not what is happening here. The right thing to say is that the regressive explanation of the boundary condition is no explanation at all.