Her and her man have talked about asking their friend to join them but she's got concerns about possibly ruining a friendship