DD1 has always drunk a lot of water. She used to have 1-2 bottles of chilled water at that age so I would say it is ok I also drink a lot of water.
She might also just like the cold sensation in her mouth because she is so hot? You can put some water in a container and some face cloths in it, then when she asks for your water but doesn't want boobie, you can give her a wet facecloth to munch on. Like an adult chewing ice to cool down