I may call the police depending on how vulnerable and isolated I felt at that particular moment in that location.
As others have said, I doubt the man stopped the swing in a friendly agreeable way.
Who knows how quickly it could have escalated? Or how many times he has used this behaviour or may use it again.
I disagree that the woman should have shared just because this freaky guy demanded the swing. He can wait and teach his child patience at the same time. 5 minutes isn't a long time.