I don't think you are overreacting as such. I can see why you're disappointed, but perhaps your disappointment is a bit premature. He is still only 4 and his parents may be control freaks who don't think anyone else can do his routine or handle him as well as they can.
My DD is 4.5 and I don't want her having sleep overs at family's houses yet as she's not done so from a young age and she's the one who is very particular about her night time routine .
I really want my nephews to be close to DH and I and I'm more than happy to look after them for an hour or so here and there or go to their concerts or sports finals matches, birthdays etc, but not much more than that. But that's just DH and I, so credit to you and your DH for wanting to be so involved in their lives.
I'd also say from that perspective perhaps his parents don't understand your desire to have such a close bond with him that's why they keep saying no?
I do agree with others who have said be specific about a specific outing and maybe when asking explain you're super keen to do so as you want to get to know him better and become closer to him. They may then be able to more understand where you're coming from.
If that doesn't work I think unfortunately you can't 'force' (not that you're forceful but I mean more so you may need to give up) the issue, you may just need to accept that's not how his parents like to operate. Their kid, their 'rules' unfortunately.
Good luck sorting it out! ❤️