Fortnightly sounds good. But please don't get in the mindset that 'he gets more than most fathers' and 'after what he has done he cant complain' these are issues you have with him not your son. So although your thinking about your son, it's great how involved he wants to be etc. many mothers beg fathers to be in the picture and care enough.
Maybe 1 weekend his house and then the other he comes to you for a few hours for a park etc or even for the night time routine etc. your doing well, situation sounds sticky!
Mediation is the next step but also consider his feeling as a father.