I live in Leichhardt with a 1 year old and 2.5yr old and it's great. There are so many playgroups (once you find out about them as they aren't really advertised but I found most through word of mouth) and lots of things within walking distance or public transport. I never drive anyway now, always use public transport as its so easy and quick and am so much fitter for walking so much.
Rent is fricking expensive though! You can get something for up to $750 but it would be prob 2 bedroom and competition for houses is very high. But I would def recommend inner west.