I agree with others, if he was hungry he would wake.
I got told some bad advice that I'm disregarding from my paed. As a mum I think you know your child best and what's good or bad advice. The paed said my 6mo baby needed a "drastic diet overhaul", and that I should limit her to 3 x 150ml feeds a day. She's 10.5kg and 74cm, so a big baby but off the charts by the same amount for birth height and weight. I think putting a 6mo on what is essentially a diet is ridiculous.
You have the opposite problem. Some people can't accept that not ever baby fits the stereo type mold. Some are big, some are small.
Go with your mummy instinct.