We took things, but I kept them with me if DD wasn't playing with them. They weren't brought so other kids could use them, but instead of tempting them by leaving it laying around, I just kept it with me instead.
If the child was only young, then I think you're overreacting. It likely doesn't understand that other people won't touch its stuff just as your child doesn't understand he shouldn't touch things that don't belong to him.
Both parents should probably make sure their kids behave properly if they're both quite small.