If my kids (5 and 3) were in the room I definitely would have spoken up. They hear everything at the moment and I would need them to know that that language is not acceptable.
I have some youngish staff in my business that I have pulled up repeatedly for saying 'gay' in place of 'lame'. Totally not on, they don't dare say it now.
Edit - just noticed you said kids in bed sorry.