It's hard to believe that a nondescript high-rise on Bloor Street between Huron and St. George was once the city’s most contentious piece of real estate.