Fan interest is starting to wane with each passing week