I'd go but that's me. I leave the house the day I get home. I can't stand being stuck at home with a baby.
My first mothers group was the most amazing group of mums. You'll find heaps who are struggling, paranoid and just a little bit OTT, as well as the chilled mums.
Have you asked how old the other babies will be?