My DH used to always find it so incredibly offensive when he'd take the kids out and people would say things like "Oh that's nice, giving Mum the day off hey?"
and he'd say- "actually no. I'm a parent too, this is what we do."
So no- I would not feel uneasy with a male childcare worker. I wish there were more of them!