I am interested to know.
I say its not your business to people and dont usually say it as an insult.
More as, its not your business so dont worry, move on, let it go.
I said it to DH this morning as he was trying to watch the neighbour have a fight with her gardener.
Its not his business.
I am surprised to see that it would be just cause to be insulted, when I simply see it as a turn of phrase with no personal insult attached?