My opinion has changed after reading this re hysterectomy. A hysterectomy is a serious op that can reduce sexual function, pelvic stability, reduce blood flow to ovaries, cause bladder problems, it's huge. It's not just a piece yo can remove, the uterus and all the associated structures that have to be cut during hysterectomy (arteries, ligaments) is a central piece of female anatomy. My mother has so many problems that are caused by hers. I think you should research the effects of hysterectomy.
Other than that, I don't know how old you are, but perhaps you and your husband could agree to reevaluate in 2 years. Financial, emotional, etc positions all can change and it may be that one or other of you may find in 2 years that you are more able to come to the others position.