My DS wasn't fixing and following at 3 months old and it seemed like he was almost avoiding eye contact when we played with or held him. It was more noticeable as his cousin who is 2 weeks younger would really stare at people and loved eye contact.
We talked to our MCHN who referred us to our GP neither knew what was wrong. It was a very scary time thinking of all the worst case scenarios.
It turned out he had a condition called delayed visual maturation which basically means everything was present and correct it just hadn't started working yet. A bit like walking later than other kids. At 6 mths he had normal vision for his age.