In your situation I would probably just take him with you if you don't want to leave him with anyone.
We are lucky to have family and friends close by who were happy to babysit for us when we wanted/needed. Though neither of my kids would take a bottle so we didn't leave them with anyone until they stretched out feeds long enough for me to leave the house for a couple of hours. That being said, our couple time has reduced significantly since having kids as we usually head out with friends on our nights 'off'. Our relationship has definitely suffered as we haven't put in the effort we probably should have.