Its really really good that you are surrounded by people that support you And it sounds like you have lots of plans in place for accommodation, uni etc. To be completely honest, I am really surprised to learn that you are in your early 20's. You always come across here as very level-headed and mature
Don't let your brother confuse you. He is coming at it from the other side, where he has completed his studies and is ready to start his career, so its kind of natural he would have this viewpoint. Do you have someone you trust, who you can talk these doubts over with? There is no harm in considering the negative side, its best to go into it fully informed & with your eyes open. But ultimately only you can decide what is right for you.
Oh and re your comments about the party lifestyle, being irresponsible etc... I partied all through my late teens & twenties, was completely irresponsible, didn't look after my diet or health etc. And I totally regret it now. It was a complete waste of time, resources and brain cells. Just thought I would give that viewpoint