I just incurred the diplomatic cost instead of them... It might also depend on your strength.
Surely people didn`t expect the `Get off my lawn` to be guaranteed that they`d go. I told a faction to get off my lawn and he said, to paraphrase, "We`re not going to go all the way round to get where we want. Be less paranoid!"
A few turns later he declared war, so I was right to be paranoid. Point is, he obviously felt strong enough to ignore my warning and even if he wasn`t, he was ready to go to war with me anyway. Why would he bother listening to me?
They will only go away with a, They want peace. B, are scared of you.
Seems to be working as intended so far.