My DH is the same OP. Unless we have some specific planned social occasion he just wants to do his own thing on the weekends. Drives me insane! Actually, if we could get someone to watch the kids he would be super happy to go out as a couple, but never wants to go out and do stuff with all 4 of us.
And the need for 'me time' doesn't cut it for me either as he goes out two nights a week for darts and soccer.
I don't know how to fix it!