OP, are you in a truly committed relationship? No issues, wonderfully happy to be having this child together?
To me, and I could have read it wrong, I get the sense you aren't entirely happy with your current arrangement and the fact he doesn't want to get married. If this is true then it would explain your reluctance & I would agree, give bub your surname.
There's no right or wrong answer, you need to do what works best for your little family. However I do believe not giving the baby his surname just because he doesn't want to get married, is not fair. Not everyone wants to be married, especially if they have already been so. Doesn't make them any less committed to their partner or children (assuming your relationship is happy & healthy).
Also, he doesn't need to worry about the ex changing his children's surname. She can't do that without his consent while he is still actively involved in their lives - no matter what she threatens.