Yes. I don't see it as anything to "forgive". He hasn't actually done anything wrong - just has different feelings and desires than you.
I don't want a second and my DH does. It's been so hard for him and of course I have felt bad, but ultimately I need to do what's right for me, our DS and our family unit. And having a second would not be good for us, as it's just something I don't want (and would end up doing vast majority of parenting as he works long hours and we have no family here).
We have talked through it a lot but I have certainly never begged forgiveness. And he would never leave me for the sake of a hypothetical second child. He loves me and DS more than that.