You sound like me, an introvert who dislikes group social settings! I'm always the shy & quiet one at social gatherings or work meetings. People who don't know me very well think that I'm rude - but I'm not! I'm just an introvert
If you aren't enjoying playgroup, then just stop going. If your DS is a little one, then he probably won't miss it. Maybe enrol him one day a week at kindy? My DS started going at 2 years old & he's formed his own friendships there.
I stopped going to the playgroup I joined & nobody gave a hoot! Some people started de-friending me on FB because I stopped going & I got removed from their FB Playgroup Group I thought, "pfft whatever!"
I have two really good Mum friends that I see about once a month. They don't know each other, I see them individually. I love the company of one on on friendships more than groups.