No I don't really worry about it. Some decisions I make are for her and some are to suit me, I don't think this is a problem. I believe that any child brought up in a loving home should turn out to be a relatively well-adjusted adult. I don't think this requires the parents' lives to revolve around their children. In fact, I think my children will benefit from learning that the world does not revolve around them.
I don't identify as an 'attachment' parent, though I know my child has healthy attachments, and I know that her emotional and physical needs are met. I don't subscribe to the idea that there is one ideal way to parent (AP or anything else). I actually think that the human psyche is much stronger than we give it credit for - I don't think it's terribly easy to f up a child, certainly not when you are parenting with love.