People have different friendships and relationships with others ... depending on individual personalities. What I would tell my bestie or my sister is different to what I share with him, or advice I ask. Doesnt mean I am cheating or dont love my husband. Just means I have other friends.
I honestly dont see a problem. Yes, it can hurt that you are not his only confidante .... but that doesnt mean he is having an affair.