If people are fungible—can be exchanged for exact duplicates without this making any significant difference of value—then any two situations where there is the same number of exactly similar people are equivalent in value. After all, we can get from one situation to the other by just replacing the people in the one situation by the people in the other.
But suppose there are infinitely many exactly similar happy people and every second one ceases to exist. Obviously, something bad has happened—a lot of value has gone out of the world. But the number of exactly similar happy people is the same. So people are not fungible.