I don't see it as 'holding them back'. I see it as exercising the movement I have to send them at 5 or 6 and deciding 6. You exercised your judgment to send at 5. Every teacher I've discussed the topic with, both personally for my kids and professionally as a pre service teacher, has clearly said they support later starts.
Only on BH, where parents who make a decision which falls into line with research all over the world are criticised lol