Chip Towers joined the show and explained the decision of eight or nine conference games may not be that simple