Labels Are Not Who You Are