Locals say they are facing serious issues with housing, jobs and harassment, even though pet-eating rumours are false.