People will treat you the way you allow them to. It is up to you to communicate effectively and tell the offender clearly about how you feel and what you expect from him.

This does not mean to offend him or pick a fight. It only means communicating assertively and not aggressively or passively.