Four year old children shouldn't be sitting in a classroom. Just because they are academically ready it doesn't mean they are ready in other aspects of their development. And it's not the first year of school that has to be considered, it's high school, puberty, etc.
If you look at the highest ranking education systems of oecd countries around the world, every country in the top 10 has a starting age of at least 6 and some with a starting age of 7 or 8. Australia currently ranks 14th.