I'd be guided by your teacher/school. They'll do a school readiness interview around enrolment time and then give you feedback on that. If they think that he's ready to start (in conjunction with his current kindy/daycare teacher's advice) then I'd go ahead. If they suggest holding back then follow that.
Its also very different state to state. In Qld its less common to hold back - DS was in prep last year and the majority of the class were 4 turning 5 (or just turned 5) with only a couple of kids a year older. I have family in NSW and VIC and they have said that its the opposite in their experience - most people start their kids a year later so that becomes the norm.
But ultimately, I'd be heavily swayed by the school's advice on this issue.