Having more children does, on average, result in a larger environmental impact. I think that's pretty common knowledge. It's just a fact. There aren't many 'what if's'.
Of course there will be exceptions.
Of course these children have the potential, like any other children, to grow up and do amazing things in this world. (I really don't understand how that's relevant to the conversation?! I mean theoretically so could all babies who are aborted...)
Of course people should have the right to have as many children as they like, no one has suggested otherwise.