Adelle ended her conversation with Australian Journalist Matt Doran when he said he hadn't listened to her album