I could have written this EXACT post OP. My DD is 6 months now and I still feel the same.
I never had a problem with my DH's family I have never been close to them or had them constantly wanting to visit UNTIL DD came along and that just made me really annoyed that they never bothered before.
When DD was a newborn I just wanted to be at home with her establishing breast feeding and just cuddling her and making sure she was happy. I NEVER ask to hold other peoples babies as it is not something that interested me and whenever I HAD to pass my newborn over to someone I could see how much she hated it and got upset so I started just saying she's not happy at the moment or something like that.
I know not everyone would agree with this. but I cant help the way I feel I did try but it only upset me.I became extremely protective of my DD although in saying that around my family it was different because they cared to ask how I was doing and didnt just rush straight to the baby forgetting about me.
DH's family never let us know when they are coming just think its okay to drop in even after DH telling them to call/message first so in the early days we just wouldn't answer the door. It's full on with a newborn and the last thing I wanted was to feel like I had to entertain.
Just do what makes you happy and don't care what anyone else says about it all that matter is that you are happy for your little baby.