Oh wow, seriously don't stress about it. I haven't had a drink since I've been pregnant, but only because I haven't felt like one, and no other reason. I know two people who'd have a glass a couple of times a week throughout their pregnancy with no problems (and plenty who would have one every now and then). In fact, looking at babies that were born around that time, those two who drank regularly have (co-incidentally I'm sure) the most alert, happiest babies out of everyone.
It takes a lot of alcohol to cause a problem with the baby. If you were downing a bottle a day, then I'd be concerned.