The baby can't defend himself. His mother isn't defending him. Someone needs to. Report it, and follow up. I reported some old neighbours once, and never saw anything happen. They eventually moved and I always wonder what happened to that little girl.
The maternal grandparents sound supportive. They will more than likely recieve custody.