I don't let it go, but I do direct my comment at my child, ie "that isn't very good sharing is it?"
If I see a child smack mine, I will say "oh don't smack, that hurts" and leave it. If another parent makes a comment I will say "Well you should be telling them this but you didn't. Is it ok to smack?"
Or if they said my child hit first I will say "well obviously I did not see that or I would have said something. Why didn't you?"