Only if you didn't feel comfortable sitting inside where the dog is. And only because I never ever let my dog outside unattended (even with a fully fenced yard) because I lost one of my precious dogs through carelessness (someone left a small access gate open, my dog wandered out and was killed by a car). I'm super paranoid/anxious about safety.
Having said that though, it's not like I'd make you sit outside on your own and talk to you through the door! I have a quite a nice (poop free) yard and a pretty patio, it's lovely sitting out by the pool with visitors.
If it were such a big issue though, we just wouldn't have visits at my house. I'd rather visit with people at a location where everyone felt comfortable.