You have to chase them away.

“And David said in his heart, ‘I shall now perish one day by the hand of Saul.’”
- 1 Samuel 27:1 (KJV)