Consider these two candidates for fundamental physical equations:
There are other kinds of beauty of physical hypothesis that do not have much to do with simplicity. Sometimes, for instance, a given physical hypothesis can be characterized in two different ways: say, using a variational principle and a mechanistic story (Leibniz often talks about this). Physicists and mathematicians love this sort of thing. It definitely contributes to the felt beauty of the theory, and a theory that has such a dual characterization will, I think, be preferred to one that does not.
We like theories that tell a compelling story. There was something very compelling about Newton's idea that force is the rate of change of momentum and that the force of gravity drops off precisely in proportion to how "spread out" it is over a spherical shell at a given distance (i.e., the force of gravity is inversely proportional to the distance).
These are all aesthetic judgments, ones like those we employ when judging a piece of art or literature. "This really goes with that." "That's just a pointless plot twist."
This could lead us to non-realism about science. But I think it is better to see a tie between the physical world and our aesthetic judgments. It is, for instance, exactly the kind of tie we would expect if the world were the work of an artist whose tastes are not utterly alien to us.