I wouldn't let my 4 or 6 year old play out on the road unsupervised and I would worry if I saw my neighbors four year old out there by himself. But that doesn't mean I think all children should be locked away.
I agree with you op. I would be more annoyed with the parents and children's lack of respect for other peoples property. Unfortunately these people will probably never change there negative attitude.