I only had my dad visit in the hospital which was all I was comfortable with. We were having attachment issues with DS so my boobs were out most of the time, and I was exhausted and feeling fugly anyway.
When we got home I felt it was quite overwhelming. DS was feeding every 2 hours for 45 minutes, but then wouldn't let me put him down for sleeps, or anytime actually. So we had a few visitors but only close family. My mum and sister visited from interstate but stayed in a motel to give me space which was fantastic, because I just wanted to rest when DS rested and wasn't up to entertaining.
After 2 weeks we tried to get out of the house for a walk and some fresh air, but DS just screamed in his pram.
We had a bigger "wet the head" when DS was 12 weeks when I felt more confident, breastfeeding was going ok, and his vaccinations were up to date.
I think it depends on the baby too. My friends have had relatively "easier" babies who sleep on their own or can be left in a bouncer awake. I'd imagine my life would've been a little easier in the earlier days if DS could've done the same.