Children don't need ponies and private school. They need food, shelter and loving parents that support them to reach for their dreams. I think you and your husband are wrapped up in money, and wealth being the answer to happiness. It's not.
You say he'll be a loving caring father... are you sure? From what you've written he's great..... when he's getting his own way It sounds like the moment you stop doing what he wants you to, and want something he doesn't, he becomes abusive. He sounds very immature to me. What happens in 2 years if you terminate? I'm going to take an educated guess there will be another excuse that you guys aren't financially 'ready'. I know someone in an almost identical situation to you. He continued to shift back ttc over 3 years beyond the agreement, for her to find out he never wanted kids and just told her that to keep her there.
You are putting your eggs all in one basket here OP.