Hi op I agree with @fearless leader- sometimes depression manifests differently to crying and moping around.
I think your DH is depressed - male PND and I think the swearing is him being frustrated that he can't help.
My DH hates it when our 10 week old fusses and cries and he can't settle her- he appears to get angry, he doesn't swear but I can see such similaritys.
Your DH may be over tired, sleep deprived, and feeling like he is powerless to help you- he loves you very much and it upsets him to see you struggling with the children and he is very frustrated that they are doing that to you.
Anyway that's just my opinion from the posts you've written and that you said how you've discussed his swearing and that he agrees it's wrong.
He needs hugs and possibly medication and counselling and a break- time out. A meal out with you And you undivided attention would do him wonders. I know he's supposed to be an adult and behave like one but it sounds like he is just not coping and he is reverting to acting like a naughty child- when my DH and I had similar problems he was really calling out for positive reinforcement from me- he wanted lots of compliments and reassurance that he was needed and that I just didn't want his sperm to get babies, I actually wanted and needed him as a person. Anyway that's just what worked for us. I really hope you can get to the bottom of it and get your DH enjoying parenting and the wonderful time it can be.
Pm if you want to. Hugs.