Joe claims he can only stay "dolled up" or dressed up (in a nice suit) for a short time on special occasions