I have been there done that,except it was ME refusing to see DH's family.
I still do sometimes,just go out to the gym when they vist EVERY sunday
Sometimes ppl just do not get along!! Your DH sounds cool,and my DH doesnt care that I dont always wanna hang with his fam
Honestly,just tell them the truth.Everyone doesnt have to love each other...as long as the kids arent missing out I dont see the prob?