Credits:

Music Track Title: Everybody Loves a Catastrophe; Licence type: Standard Licence; Artist: Twirl; Composer: Neal Busby & Huguette Arsenault (SOCAN); Publisher: Lynne Publishing (PRS - CAE#: 541626758)

Logo Artist: Avril Orloff; Outsidethelines.ca

Transcript:

Welcome to the podcast. Risk is the new kale. Each episode I talk with folks who have figured out how to extract opportunity from risk. As someone who has spent a career controlling risk, I want to know those who embrace it. Risk is the new kale good for you? Hard to take.

Doctor AJung Moon is remarkable. She chose a field of study that quite simply no one was looking at at the time. This podcast examines the issues of potential conflicts between humans and machines from an expert in responsible design of robots and artificial intelligence. Ajang and I first met when she launched a consulting company called the Open Robotics Institute in order to offer ethics reviews of organizations deployment of AI. To say that she opened my eyes is an understatement. And to this day the work that she and her team did instilled the language of AI ethics into that particular organization's DNA.

OK, here we go. She is an experimental roboticist specializing in ethics and responsible design of interactive robots and autonomous intelligence systems. A professor at the University of McGill, she founded the Open Robotics Institute and she has advised the UN Secretary General's High Level panel on digital cooperation as well as the government of Canada's Council on Artificial Intelligence. Welcome to Doctor AJung moon. Hello, thanks for having me. It's great to have you OK adjunct. What is a robo ethicist? This is a job title that really didn't exist five years ago. Yeah, I think I just made my own job title. And I think that the ruble athesis is still a term in making. Robert, this is essentially means people people like myself who may be robotics is really looking into ethics and trying to understand the ethical implications of the things that we build and try to bring it up, but not so consider the ethical implications during the design phase. And there are also robot SSS or people who I would qualify as Robot FSS who are actually studying the ethical implications on the whole of robotics, generally speaking. Even you know, even if they may not be designing the robots themselves. Yeah, I love that you believe technology is a superpower for humans. So what is the role of autonomous

systems in society? You're asking a very tricky question here because I think I think technology is still. I still believe that technology is a superpower because I think we can do so many interesting and powerful things using technologies that we were never able to do so before.

And at the same time we are in this era where we can start to think about machines that are autonomous and intelligent and they can do massively scalable things for us so. If I were to say technology, broadly speaking is a superpower for humans, I would say autonomous systems are like superpower plus plus, it allows us to do things much faster, much more in in mass in ways that we never envisioned before. But in terms of the role of autonomous systems in our society, I think that's something I'm still trying to figure out. Because I don't think we have a very close ended answer on that question. Well, that I think that bounces to the next question I have for you, which is societies perception of risk associated with robotics and autonomous systems. There's sort of this perception. Maybe that's maybe quite different from the reality. Can you talk a little bit about that? Yeah, when we when we talk about risk and robots, I think traditionally we were thinking about oh robots working around robots can be dangerous. We want to separate robots out from humans because what if robots end up punching you? That's the physical risk and that's something we can really wrap our heads around and put some barriers on top. But then there's also the sci-fi notion of

risk with respect to robots. Which is things like Terminators. We might have this great robots that we trust, and we have all over the world and maybe all of a sudden they will just turn on us. And that's how things will go bad. But I think that the kind of risks that I study is really more about the risk that come from the interaction between humans and robots coexisting in a space.

And it's not necessarily like a Terminator that will. That is out to get you, but it's really more about how much of faulty things will we end up believing because of robot is telling us this particular information. So really changing almost the way that we interact with our world, and much less about a physical safety. It's more kind of shifting societal behavior in a way is what you're looking at.

Yeah, I think there's a lot that I'm discovering about human behavior. Some of our psychological effects or our behaviors can be predicted because a great example could be a robot that moves super fast, as though is punching you just about to punch you, but it's program not to punch you. It will stop a few millimeters in front of your face. You probably will. Respond without even thinking about it by blinking, maybe ducking, maybe shying away and having this fear response where your your palms will get sweaty and your heart rate will go up. Those are things that we probably won't be able to control. These are bodily reactions that are built into us, but by programming the robot to move in such a way, I cannot predictively elicit that response in you isn't that powerful? Yes, that's so interesting, that's. In a way kind of the managing the ethical and moral responsibility not to elicit a response that is. Is not something that we wish to respond with, but it's actually tapping into kind of the just as you say, the physiological response. That's automated because of the nature of us being human beings. That's really interesting. Yeah, part of it could be also also purely psychological as well. One really interesting study done in 2018 by Robinet Al. They were really interested in this effect of Overthrust. Will people trust robots in such a dangerous level that we need to start to worry about it and so they created this experiment where they introduce these people to a robot that they are working? And they said, oh, it has this

kind of little flag on it. We're trying to design it so that it can guide you towards a fire exit if there is any case of fire in the building. By the way, the just in case there is fire during the experiment, the exit is this way and they would give them all the safety orientation and they said because the robot is is still a prototype. It's faulty, it's not going to direct you in the right direction, so don't follow the robot and lo and behold, they fake triggered this fire alert and they watched people whether they follow the robot towards supposed exit or they go towards the exit. They have been told is the right. And they saw that majority of the participants actually follow the robot towards a dead end. Really, that's the kind of humanoid robot. Do you think that had something to do with it? No, it was just that was an instruction that they had in the immediacy. So they followed it exactly. It doesn't look humanoid at all. It's literally like a boxy cylindrical thing that moves with things sticking out of it. Nothing too sophisticated, right? So you would think like, why would humans do this? This is not the kind of risk that we. We have in mind when we build these and now we are starting to think about this interactive behaviors that are really quirky when we put robots in the room. Would you describe that level of interaction as a conflict between humans and machines?

Yeah, I, I think we can think about it as a form of conflict, but a form of conflict that we have yet to uncovered because when we talk about conflicts, I think we envision this direct kind of head-to-head kind of conflict. But in this particular scenario about the experiment with over trusting of robots. The robots don't really have intentions, they're just machines. They're just programmed to do whatever they're programmed to do, and many times they will fail as well. In that sense, we can't really frame things going wrong from this interaction, as though it is really about human human conflict, headbutting of differences, opinions or intentions or desires, because in a way a lot of it is very one sided. Oh interesting, yes. Yes, we have a lot of accountability in that as the human. Yeah, as a user, especially with the punching robot example that I gave you. What, how, how much of that interaction would I say is about accountability? How much of my physiological response, the threat response? What I say is part of my control and how much of things that I can control about myself. Can I say is something I can be accountable for? These are open ended questions. Yeah, well where do you place the accountability of the designer and the programmer of the robot? Then in that interaction did they hold the majority of the ethical accountability right now? No, I would say this is a very contentious topic. Very open-ended issue right now, especially when it comes to self driving cars and some of the accidents that we have seen. We're really looking at the limitation. Between what humans can do. And what robots are very good at and we have situations where we are getting people to monitor these systems. That majority of the times they work perfectly fine, but humans are there to catch that 10% of the time or 5% the failure modes. But if the system has been working fine for the majority of the times, you're not going to pay attention to this. You're going to be complacent about it. Automation

complacency is a behavior that we understand very well. In human machine interaction. And yet, with the kind of systems that we are seeing with human testers, we actually hold the human testers or users accountable for things going right. If they're actually not paying attention to the road and so on and so forth. Although we are really hitting the limit of human cognition at that point. And I would say that this particular contentions topic is there because our legal system that protects us from certain types of risks that we know about in interacting with other humans don't really apply well when we start to look at these machines that do a lot of interesting superpower stuff. But also there's a lot of new uncertainties for us as well. As we interact with robots as we interact with artificial intelligence systems, as we're sort of blending the environment, so law has to expand, and I just so appreciate this. Underpinning that you're bringing around the ethics side of it. I think it's important for people to also understand your definition and the difference between robotics versus artificial intelligence. Perhaps you can sort of ground us a bit in some definition. Sure, so I define robots to be these systems that have a body. That have physical, tangible things you can touch. As AI doesn't need a body to be artificially intelligent, so in a way I say that not all AI needs a physical body. All robots have a body and not all robots need AI to operate OK. OK, so for example, I think a great example is one of my colleagues, Professor Audrey Sedal, also at Miguel. She's

an expert in soft robotics and soft robots. Are these systems made of these composite or stretchy material that are? You can even actuate with a bike pump essentially so that they change shapes and they're soft and they can move in ways that are useful for us, but they don't need any kind of intelligence in the background to operate, and perhaps another example of a robot that doesn't have a. I would be robots that I've seen when I was an intern. An automotive factory many many years ago. These companies have gigantic robots that paint the chassis of a car as they come down the conveyor belts, and these systems can be programmed to repeat the same task over and over again. The next car will be blue. The next time will be red. You don't need to supervise all of that. That's all programmed in there, but nothing about it is artificially intelligent in that sense. OK.

Yeah, it's a continuum, I think is how you're describing it. And as we're now in the 22nd century, my perception of robots is that most have some kind of intelligent kind of analysis of what's going on around them. But I think you've cleared up that there can be many types of autonomous systems that are just simply doing the same tasks over and over again, like a piece of machinery in the 1700s as an example.

OK, I'm curious about your stories here around how do robots and AI systems influence us? Because you study how these systems change us and that they influence how we as humans move and behave and make decisions. Can you tell us a background story around your experimentation and your research there? Sure, I can try and scratch the surface. Eliz robots and AI. As we briefly discussed, are kind of two different beasts that work well together. I think of robots as these physical things that can be powerful influencers of our behaviors because they have the body. And they move around in such a physically salient immediate ways, whereas AI, the kind of AI that we think of the software that's running on our phones. Software behind Google Maps or any apps that you use to drive around, find the best routes from point A to point B. All of those systems I would say are systems that influence us in powerful ways because they're so invisible they can be so invisible. The part that I'm really fascinated by is the fact that even the youngest of babies three months, six months old. Show them things moving around you. Tell them stories of the circle that is triangle. And then after a little while you, you give them choices of you know the circle toy or the triangle toy and then the babies will definitely prefer the good. Triangle over the circle and that inherent. Perception we have between mapping of motions to something that is good, something that will protect us. Something that will allow us to be happy. I think that translates to how we see robots and the way they move. Robots in a way, have the capacity to be programmed

to lead us to be very anxious, feel unsafe, or really feel comforted. Is that a deliberate choice of programmers, or is it something that not all designers of systems actually think about and are intentional about it? The start of whatever they're creating? I believe it's more of the latter than the former that we're just starting to really understand the full effects of robots and their emotions, and even verbal behaviors around us. Yet we are developing these systems without having had this full understanding of them because roboticists tend to share this. Where robots will be in our homes, they'll be doing our chores. They won't get in the way. They won't interrupt us when we are busy doing things. There will be all this very intelligent and amazing things, but just the question of what does that mean for us to even have a robot in the home? What does that do for our perception of privacy? Of being in my bedroom and all of a sudden there is a robot. I think that's the kind of question we're starting to address that we don't have full answers for. So in a way, we're kind of parallel tracking between designing of these robots that could be really cool and understand the implications of it at the same time. Fascinating emerging field and as you're as you're describing it, it's helping me understand, kind of where experts like you are thinking and actually leading us to be questioning. How are we going to feel and react in the same way that I have? An intelligent speaker on my desk and every now and then I think you know what I'm about to have a conversation. I don't actually want that on. I'm going to unplug it, but

I don't always think about that. And now I'm actually quite comfortable with it sitting there. It's just a little Alexa speaker and but that's taken some time, but it's funny how quickly that integrated into just sort of the day-to-day in the household. Definitely, and even understand the longitudinal effect is something that we are starting to do more and more in, especially in the domain of human robot interaction we put on our scientist hats on and do this control experiments. Half an hour come in and work with the robots for us. But we are also seeing that when you give people these robots to hang out with for many, many weeks down the road, some people may develop a strong emotional attachment to them. Some people might not even bother with them after a couple of weeks, and some people are kind of in between, so we have yet to understand and unpack the nature of these relationships, I would say.

So you were describing domestic robots, but of course there's industrial robots, transportation, robots, and military and security robots. In what ways are their similar risks associated with them, and in what ways would you describe some extreme differences? I like to think about risks deployed in different applications separately. When we talk about domestic robots. I believe we have the capacity to think about the scalability. We will likely be mass manufacturing them. We will want to think about maintenance of them. Updates to the software that end up being massive manipulation engines for all the users.

Whereas if we start to think about robots as weapons that can be make targeting or triggering decisions autonomously, then. We're not only talking about the ethical implications of doing that or the physical harm of the people on the other side of the weapon. But we are also talking about the political risks and implications of that as well. So the scale and the spectrum of risks are, I think, quite different. But again, maybe the similarity across the board is really about the fact that risks with respect to robots are really social technological risks. We can't. We can't stop our discussions about risks in the traditional sense. That says, ah, there is. I don't know, 2% probability of failure or if the robot were to come into a physical contact, then the likelihood of you losing your eye is XI. Think our days of just talking about safety or traditional sense of risk is kind of over in terms of robots, in part because they're now. Sharing our physical world with us outside of these boundaries. So the boundary is more porous, but the opportunity is greater thus. That's why they're adjacent to us, so such a fascinating thing in that spectrum of difference all the way to the politics of deploying autonomous weaponry. You were on a UN committee that looked at the use of robotics in weaponry and warfare. Do you believe that as a sort of global geopolitical? Discussion that the right conversations are happening there. Do you have confidence that the right people are in the

room? So the UN Convention on Conventional Weapons is the international fora where we're having the killer robots discussion whether to ban such a system or whether to have a treaty and if we have a treaty, what would that look like? But I would say I'm hopeful about the fact that we're having the conversation. Because it is a, it is an International Convention under the umbrella of the UN community. The pace at which this conversation is happening is much slower than me sharing a Google Doc and Co. Drafting something with you. So it's a multi year process, and as technologists I think a lot of us share the perspective that this is way too slow. We have a lot of the members of the Community who support a ban or who support some sort of a regulatory framework for against this kind of. Advancement of military technologies and we need action now. I think we're being a little bit impatient there as well, and I'm not sure what the answer is to marry the necessary slowness of these multi state dialogue and the necessity of catching up with the pace of the technology. But when I was working as a senior advisor for a high level panel of Digital Corporation, part of what the panelists were discussing were really about that notion of when we think about digital technologies. What kind of big issues should be cool? Should we be cooperating on as an international community? Because even bringing everybody to the same table is?

Hard task to do. And in a way, I think there is a segregation between people who usually go to these UN conventions. People who hang out in the Silicon Valley and talk to the investors. And then there's also people who are leveraging on on open source code base and contributing to really critical pieces of software that the whole world relies on, but is not being paid to do any of that work is completely voluntary. Those people don't often meet in the same location, nor do they have an overlapping community, so I think that's a big problem that would be hard for us to solve, but important to solve. In the same way that agile as a philosophy has kind of permeated from software and tech into the business community, I feel the next place for that to play out is in public policy so that you trial a new policy and people agree that might not be perfect, but you roll it out. I think GDPR is an example of that. Some of that leadership where privacy and one part of the global community has just become something that is being tried and it. Seems to be influencing what's happening in North America. It gives me hope that people like you are involved. I have to say.

And John, what does it mean to embed human values into predictive data driven algorithms? Me embedding human values into predictive systems. Means understanding what values we share as a community in the first place. Having an open conversation about how the new technology that we're trying to deploy. Interact with those values. And making sure that they are aligned.

In my past life, in the startup world. I was like, yeah, I thinks consultant in part working with you in deploying this kind of process of unpacking stakeholder conversations into specific set of values and then thinking through how do Joe understand the notion of fairness differently from Joe? For example. I'm just making up names here. And why does that matter to the design of this piece of software? That may impact people's decisions down the line. That process is something that we are still continuing to develop, but it's also a very necessary process. I would say in really starting to understand what doesn't mean for us to build systems responsibly. And I say that because a lot of my students, a lot of the students who are really excited about AI and they're going to coding boot camps and so forth. They would like to have these big tech jobs secure jobs that are highly paid and whatnot. But they they may not realize that once you join this community of developers, the big homework, the challenging homework for them is to really have a conversation with their colleagues about what do they care about. Why are they doing it this way? Why is it different from what I believe in? And why is the leadership going in an opposite direction? Perhaps, and being able to reconcile that? And I think that's also a very human story of designing technology that we are. Again, still in the early stages of uncovering. You opened my eyes because

when I when I thought about the risk of deploying AI accuracy. Really sort of went to the top of what I thought was one of the most important risks. What if the algorithm was inaccurate and led to poor outcomes, but you, in describing the way that it would be used and the impact on the users and the stakeholders and the clients? Eventually you helped me understand and we saw that the users of the algorithms had a value around transparency and understanding. They really wanted to know why was the algorithm. Making the decisions that it did. Your work led to a way of daylighting. Some of the principal parameters that were enabling certain outcomes out of that algorithm gave them so much confidence too. I just so appreciate that it just was like a wow. This is amazing. Different framing. I'm so glad this is on recording right? Didn't say you were caught, you absolutely well it shifted the organization and I have to tell you that. So this is, you know years later. There was a decision by a client care group to deploy.

Sort of autonomous robotic processing. So just basically taking keyboard strokes and replicating in a very, very quick way, a way of pulling in a file, examining it, making a decision, and then sending a note back out to a user, but it's saying it was potentially going to save hundreds of hours of an individual's time. Not having to do those keystrokes. All of that was just a robotic process. The users, the employees. Because the language of an ethics assessment was now part of the history of the company, did their own assessment. Oh, and weighed the outcome as to whether this was important work that they would continue to do, or whether they were happy to hand it off to or bought it process automation. And in the end. They agreed management was not involved in this, that this was an important efficiency gain and that they were better off working on higher value work as humans. Oh my goodness. So that shifted the culture of the organization such that it took. Into the culture, we get to examine the application of an autonomous system. And we're going to make a decision. Which is amazing, like it was getting goosebumps. You did the pixels, but you totally did that. OK, I'm gonna. I'm going to jump onto a different question. What is the way that you personally assess risk like how has it changed? How you do your work over the years? By it are you referring to something specific or more broadly, about my personal risk management? Well, I see you as

you're an expert. You've made a decision to move across the country and be a prop in a different university. You you certainly put yourself out there going on these panels. You have so much confidence. I'm sure you have a ton of PhD students that you're supervising. You are living your life out loud, so you're making risk choices in your career, but also in many other aspects of your life. And part of the podcast. I think is just how do people make this decision around positive risk taking? Because you can see opportunity.

Yeah, I. I'm pleasantly surprised that you viewed me as somebody who would take risks in the positive kind of way, in part because I don't really consider myself as a risk taker. I think I'm hard on myself for not taking enough risks.

Umm? And in a way I I I I kind of have a chat with my PhD students as well, saying well, why are you asking me whether you're you you want to apply for this position or not. You should just go for it. You know what is the risk in part because I see that element in myself, which is a little bit of self doubt weighing pros and cons and being indecisive about it. But at the end of the day. I think something that has helped me in making decisions, whether I am really weighing the risks and in my personal life or not is more that I have a clearer sense of the kinds of impact I want to have in the work that I do. I think I had a consistent theme for many years that I want to be in the overlap between technology design and practically. Speaking, but doesn't mean for us to think about ethics. On top of that, and it has been itching at me as a lifelong homework that I haven't finished and somehow it has really led me to these different opportunities that presented itself even my work in working at the UN headquarters was an opportunity that came to me when I didn't expect it at all. Even the the project that I did with you was not a contract that I had expected. I had reached out to you for mentorship because I wasn't sure Katherine should I launch this nonprofit full-time or do a startup consultancy and you said, why not do both? So I think having mentors like you. Who who would ask this questions

about. Yeah, try both of them, why not? I think that that helped me along the way. That whole description is amazing and what I see is universally true of people that I'm talking to is this. External motivation and it comes to people through different means, but that seems to actually help frame the choice of decision and seeing risk as opportunity. That's my current hypothesis at the moment.

So Doctor Moon you have extensively published your research and it's been cited in publications between a 1000 times. Seriously, I just odd when I Google you and one of the publications or your research that's most cited is a paper titled Meet Me where I'm gazing, how shared attention. And shared attention gaze effects human robot hand over timing. Can you tell me about that? That particular research? Sure, perhaps this was another risk that I took without knowing that this was a risk. It's a paper that came out as a result of a hackathon project that I participated in with, I believe, about 8:00 or nine different people, and in this paper we were just really curious. Whether at this big new humanoid robot that we just got in the at the Curst lab at UBC. If we were to design the robot to act a little bit more human, like as in when it's trying to hand over a water bottle to you, it would look at different locations in space, like a human would as they're handing over the water bottle. With that actually make a difference in people who are trying to receive that water bottle from the robot. So. We were we were just thinking that would such a simple thing, such as a gaze behavior, as in where is the robot looking? Would that make a difference? And it turns out it did. It had a significant.

To tell when the robot will stop moving or when the robot will be finished handing it over to to the person. So even before the robot Clunkily reached its hand to stop moving and say please take the water bottle. It didn't even have that cue. People would intuitively know when to start to reach out and meet the robot halfway. So I think that was a really cool study that we've done we. Launches an experiment with over 100 participants that we pulled from UBC orientation week just randomly said random interact with the robot. Get a free bottle of water.

And yeah, I'm I'm so happy that this was such an impactful work in human robot interaction. Very cool, thanks for asking. This is a lot of fun too to remember that project. That's very cool. OK, my final question to you and John if you could make anything better on this planet anything, what would it be? If I could make anything better. I think I am working on my my big homework which is.

Making people be more considerate to one another in small tasks, small decisions. My homework even more scope down version of that story would be to try and make sure that when we're designing things we are being a little bit more thoughtful and considerate to each other. Yeah, that's what I'm working on. I'm not sure how far I can go in my lifetime, but that'll be my gift to the planet. Beautiful, that's fabulous. Anjung. Thank you so much for our conversation today. Thank you for having me. This was a lot of fun and thanks for telling me about the story of what happened after our project together as well because. There is really lack of documented understanding of whether doing AICS has any value. I'm at the quest to think about is there a value add to Aics because we don't have an answer yet?

Doctor AJung moons research and thought leadership can be found on her website at jongmin.com, as well as McGill University site under electrical and computer engineering. You can follow her on LinkedIn and on Twitter at Robo Ethics.