Of course you should go home now - especially considering you do not rate your husbands response to what has happened!
How old are the kids? That's a key factor. Have you considered the possibility that your husband has misinterpreted what your DD said or that you DD has miscommunicated what happened? My ds1 is 4 and still can't clearly articulate what has happened sometimes - plenty of opportunity for crossed wires.
I don't mean to be dismissive just find it really hard to believe this happened without a staff member seeing, impossible to believe that your DH sent your DD back to daycare after his happened (major wtf!) and am having great difficulty understanding why you haven't already booked your plane ticket back home.