View Details

Action from RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, will this year take place in Brazil, from 15-21 July. An important part of the week is RoboCupJunior, which is designed to introduce RoboCup to school children, and sees hundreds of kids taking part in a variety of challenges across different leagues. This year, the lead organizer for RoboCupJunior is Ana Patrícia Magalhães. We caught up with her to find out how the preparations are going, what to expect at this year’s competition, and how RoboCup inspires communities.

Could you tell us about RoboCupJunior and the plans you have for the competition this year?RoboCup will take place from 15-21 July, in Salvador, Brazil. We expect to receive people from more than 40 countries, across the Junior and Major Leagues. We are preparing everything to accommodate all the students taking part in RoboCupJunior, who will participate in the Junior Leagues of Soccer, Rescue and OnStage. They are children and teenagers, so we have organized shuttles to take them from the hotels to the convention center. We’ve also prepared a handbook with recommendations about security, places they can visit, places to eat. The idea is to provide all the necessary support for them, because they are so young. We’re also organizing a welcome party for the Juniors so that they can experience a little bit of our culture. It will hopefully be a good experience for them.

The Juniors will be located on the first level of the mezzanine at the convention center. They will be separate from the Major Leagues, who will be on the ground floor. Of course, they’ll be able to visit the Major Leagues, and talk to the students and other competitors there, but it will be nice for them to have their own space. There will also be some parents and teachers with them, so we decided to use this special, dedicated space.

RoboCupJunior On Stage at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Do you have any idea of roughly how many teams will be taking part?Yes, so we’ll have about 48 teams in the Soccer Leagues, 86 teams in the Rescue Leagues, and 27 in OnStage. That’s a lot of teams. Each team has about three or four students, and many of the parents, teachers and professors travel with them too. In total, we expect about 600 people to be associated with RoboCupJunior.

RoboCupJunior Soccer at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Have you got more RoboCupJunior participants from Brazil this year due to the location?Yes, we have many teams from Brazil competing. I don’t know the exact number, but there are definitely more Brazilian teams this year, because it’s a lot cheaper and easier for them to travel here. When we have competitions in other countries, it’s expensive for them. For example, I have a team here in Salvador that qualified for the super regional event in the US and our team couldn’t go. They had qualified, but they couldn’t go because they didn’t have money to pay for the ticket. Now, it will be possible for all the Brazilian teams qualified to participate because it’s cheaper for them to come here. So it’s a big opportunity for development and to live the RoboCup experience. It’s very important for children and teenagers to share their research, meet people from other countries, and see what they are doing, and what research path they are following. They are very grateful for the opportunity to have their work tested against others. In a competition, it is possible to compare your research with others. So it’s different from conferences where you present a paper and show your work, but it’s not possible to compare and evaluate the results with other similar work. In a competition you have this opportunity. It’s a good way to get insights and improve your research.

RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Your role at this RoboCup will be organizing RoboCupJunior. Are you also involved in the Major Leagues?Yes, so my main role is organizing RoboCupJunior and I am also one of the chairs of the RoboCup Symposium. Besides, some teams from my laboratory are competing in the Major leagues. My team participates in the @Home league, but I haven’t had much time to help them recently, with all the preparations for RoboCup2025. Our laboratory also has teams from the 3d Simulation Soccer League, and the Flying Robots Demo. This will be the first time we’ll see a flying robot demo league at a RoboCup.

We’ll also have two junior teams from the Rescue Simulation League. They are very excited about taking part.

RoboCupJunior Rescue at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

RoboCup was last held in Brazil in 2014, and I understand that there were quite a lot of new people that were inspired to join a team after that. Do you think the 2025 RoboCup will have the same effect and will inspire more people in Brazil to take part?Yes, I hope so. The last one inspired many, many students. We could perceive the difference before and after RoboCup at that time, related to projects in schools. In 2014, RoboCup was held in João Pessoa, a city in the north east that is not as developed or populated as many other states in Brazil. It really improved the research in that place and the interest in robotics especially. After the 2014 RoboCup, we’ve had many projects submitted to the Brazilian RoboCup competition from that state every year. We believe that it was because of RoboCup being held there.

We hope that RoboCup2025 next month will have the same effect. We think it might have an even bigger impact, because there is more social media now and the news can spread a lot further. We are expecting many visitors. We will have a form where schools that want to visit can enroll on a guided visit of RoboCup. This will go live on the website next week, but we are already receiving many messages from schools asking how they can participate with their group. They are interested in the events, so we have high expectations.

We have been working on organizing RoboCup2025 for over a year, and there is still much to do. We are excited to receive everybody here, both for the competition and to see the city. We have a beautiful city on the coast, and some beautiful places to visit, so I recommend that people come and stay for some days after the competition to get to know our city.

About Ana Patrícia

| | Ana Patrícia F. Magalhães Mascarenhas received her PhD in Computer Science from the Federal University of Bahia (2016) and Master in Mechatronics from the Federal University of Bahia (2007). She is currently an adjunct professor at the State University of Bahia (UNEB) at the Information Systems course. She is a researcher and vice coordinator of the Center for Research in Computer Architecture, Intelligent Systems and Robotics (ACSO). Her current research focuses on service robotics and software engineering, especially related to the use of Artificial Intelligence (AI) in the software development process and in Model-Driven Development (DDM). |

View Details

Claire chatted to Gabriel Skantze from KTH Royal Institute of Technology about having natural face-to-face conversations with robots.

Gabriel Skantze is a Professor of Speech Communication and Technology at KTH Royal Institute of Technology. He specializes in conversational systems and leads several research projects on conversational AI and human-robot interaction. His work focuses on computational models of spoken interaction, integrating both verbal and non-verbal aspects such as prosody, turn-taking, feedback, and joint attention. In 2014, he co-founded Furhat Robotics, where he continues to serve part-time as Chief Scientist.

View Details

The Salvador Convention Center, where RoboCup 2025 will take place.

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event, where teams gather from across the globe to take part in competitions across a number of leagues, will this year take place in Brazil, from 15-21 July. We spoke to Marco Simões, one of the General Chairs of RoboCup 2025 and President of RoboCup Brazil, to find out what plans they have for the event, some new initiatives, and how RoboCup has grown in Brazil over the past ten years.

Marco Simões

Could you give us a quick introduction to RoboCup 2025?RoboCup will be held in Salvador, Brazil. When RoboCup was held in Brazil 11 years ago, in 2014, we had a total of 100,000 visitors, so that was a great success. This year, we expect even more, around 150,000, during all the events. Nowadays, AI and robotics are attracting more attention. We are also in a town, Salvador, with a bigger population than the previous location (João Pessoa). For these reasons, we estimate the attendance to be about 150,000 people.

Regarding the number of teams, registration has not closed yet, so we’re unsure about the final numbers. However, we expect to have about 300-400 teams and around 3000 competitors. We have been helping with visas, so we hope to see higher participation from teams who couldn’t attend in the previous two years due to visa issues. We are doing our best to ensure people can come and have fun at RoboCup!

This is also a great year for the RoboCup community: We have just agreed on new global league partners, including the Chinese companies Unitree, Fourier, and Booster Robotics. They will bring their humanoids and four-legged robots to RoboCup. These will not only be exhibited to the public but also used by some teams. They are amazing robots with very good skills. So, I think this will be an amazing edition of RoboCup this year.

Did the 2014 event in Brazil inspire more teams to participate in RoboCup?Yes, we have seen a significant increase in our RoboCup community. In the last two years, Brazil has had the fourth-largest number of teams and participants at RoboCup in Bordeaux (2023) and Eindhoven (2024). This was a very big increase because ten years ago, we were not even in the top eight or nine.

We’ve made a significant effort with RoboCupJunior in the last ten years. Most people who’ve taken part in RoboCupJunior have carried on and joined the RoboCup Major League. So, the number of teams in Brazil has been increasing year by year over the last ten years. This year, we have a great number of participants because of the lower travel costs. We are expecting to be in the top three this year in terms of the highest number of participants.

Photo of participants at RoboCup 2024, which took place in Eindhoven. Photo credit: RoboCup/Bart van Overbeeke

It’s impressive that so many RoboCupJunior participants go on to join a Major League team.Yes, we have an initiative here in Brazil called the Brazilian Robotics Olympiad. In this event, we chose two Junior Leagues – OnStage and the Rescue Line League – and we organized a competition based on these two Leagues. We run it in regional competitions all over Brazil – so 27 states. We organize at least one competition in each state during the year, and the best teams from each state come to the national competition together with the Major Leagues. We organize the Brazilian Olympiad to get RoboCupJunior to more students. This is how we’ve managed to increase participation in RoboCupJunior. Then, when students go to university, many of them continue to participate, but in the Major Leagues. So that’s a very successful strategy we’ve used in Brazil in the last 10 years.

Could you tell us about some more of the highlights from the Brazilian RoboCup community in recent years? Two or three years ago, one of the Brazilian teams was the champion of RoboCup @Home. We have seen a big increase in the number of teams in the @Home League. In the national competition in Brazil, we have more than 12 teams participating. Back in 2014, we only had one team participating. So we’ve had a great increase—this League is one of the highlights in Brazil.

More teams are also participating in the Small Size League (part of the soccer League). Two years ago, one of the Brazilian teams was the champion of the division B of the Small Size League. So, over the last five years, we’ve seen some Brazilian teams in the top three positions in Major Leagues in the RoboCup world competition. This is a result of the increase in the number of teams and the quality of what the teams are developing. So at this time, we have an increased number of publications and teams participating in the competition with good results, so that’s very important.

Another excellent contribution for this year is a league we created five years ago – a flying robot league, where autonomous drones perform some missions and tasks. We’ve proposed this League as a demo for RoboCup2025, and we will have a Flying Robot Demo at the competition this year. This will be the first time we’ll have autonomous drones at the RoboCup competition, and the Brazilian community proposed it.

RoboCup @Home with Toyota HSR robots in the Domestic Standard Platform League, RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Will you be taking part in the competition this year, or will you be focusing entirely on your role as General Chair?This year, my laboratory (ACSO/Uneb) has qualified for the 3d Simulation League (soccer), the Flying Robot Demo, and RoboCup @Home, so we are participating in three Leagues. We also supervise RoboCupJunior Teams in the Rescue Simulation League. This year, my students have had only a little supervision from me because I’ve been very engaged with the organization.

In our 3D simulation team, we have lots of developments with deep reinforcement learning and some new novel strategies that allow our teams to gain new skills, and we are combining the new skills with our former multi-agent coordination strategy. For this reason, I think we will have a robust team in the competition because we are not only working on skills, we are also working on multi-agency strategies. When both aspects are joined, you can have a really good soccer team that plays very well. We have a good team and expect to achieve a greater position this year. In the latter years, we were in the top four or five, but we hope to get into the top three this year.

In 3D, you not only work on multi-agent aspects but also need to work on skills such as walking, kicking, and running. Teams are now trying to develop new skills. For example, in recent years, our team has developed the sprint running movement, which was a result of deep reinforcement learning. It is not a natural running motion but a running movement that works according to the League’s rules. It makes the robots go very fast from one point to another, making the team very competitive.

Most teams are learning skills but don’t know how to exploit them strategically in the game. Our focus is not only on creating new skills but also on using them strategically. We are currently working on a very innovative approach.

This year, the simulation league will run a challenge using a new simulator based on MuJoCo. If the challenge goes well, we may move to this new simulator in the following years, which can more realistically simulate real humanoid robots.

Action from the semi-finals of RoboCup Soccer Humanoid League at RoboCup 2024. Photo: RoboCup/Bart van Overbeeke.

Finally, is there anything else you’d like to highlight about RoboCup2025?We are working on partnerships with local companies. For example, we have sponsorship from Petrobras, one of the biggest oil companies in the world. They will discuss how they are using robotics and AI in their industry. They were also one of the first sponsors of the Flying Robots League. It’s important to have these links between industry and the RoboCup community.

We also have excellent support from local companies and the government. They will be showing the community their latest developments. In the Rescue League, for example, we’ll have a demonstration from the local force showing what they do to support people in disaster situations.

This event is also an excellent opportunity for RoboCuppers, especially those who have never been to Brazil, to spend some days after the event in Salvador, visiting some tourist spots. Salvador was the first Brazilian capital, so we have a rich history. There are a lot of historical sites to see and some great entertainment options, such as beaches or parties. People can have fun and enjoy the country!

About Marco

| | Marco Simões is an Associate Professor at Bahia State University, Salvador, Brazil. He is the General Chair of RoboCup2025, and President of RoboCup Brazil. |

View Details

In this interview series, we’re meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. The Doctoral Consortium provides an opportunity for a group of PhD students to discuss and explore their research interests and career objectives in an interdisciplinary workshop together with a panel of established researchers. In this latest interview, we hear from Amar Halilovic, a PhD student at Ulm University.

Tell us a bit about your PhD – where are you studying, and what is the topic of your research?I’m currently a PhD student at Ulm University in Germany, where I focus on explainable AI for robotics. My research investigates how robots can generate explanations of their actions in a way that aligns with human preferences and expectations, particularly in navigation tasks.

Could you give us an overview of the research you’ve carried out so far during your PhD?So far, I’ve developed a framework for environmental explanations of robot actions and decisions, especially when things go wrong. I have explored black-box and generative approaches for the generation of textual and visual explanations. Furthermore, I have been working on planning of different explanation attributes, such as timing, representation, duration, etc. Lately, I’ve been working on methods for dynamically selecting the best explanation strategy depending on the context and user preferences.

Is there an aspect of your research that has been particularly interesting?Yes, I find it fascinating how people interpret robot behavior differently depending on the urgency or failure context. It’s been especially rewarding to study how explanation expectations shift in different situations and how we can tailor explanation timing and content accordingly.

What are your plans for building on your research so far during the PhD – what aspects will you be investigating next?Next, I’ll be extending the framework to incorporate real-time adaptation, enabling robots to learn from user feedback and adjust their explanations on the fly. I’m also planning more user studies to validate the effectiveness of these explanations in real-world human-robot interaction settings.

Amar with his poster at the AAAI/SIGAI Doctoral Consortium at AAAI 2025.

What made you want to study AI, and, in particular, explainable robot navigation?I’ve always been interested in the intersection of humans and machines. During my studies, I realized that making AI systems understandable isn’t just a technical challenge—it’s key to trust and usability. Robot navigation struck me as a particularly compelling area because decisions are spatial and visual, making explanations both challenging and impactful.

What advice would you give to someone thinking of doing a PhD in the field?Pick a topic that genuinely excites you—you’ll be living with it for several years! Also, build a support network of mentors and peers. It’s easy to get lost in the technical work, but collaboration and feedback are vital.

Could you tell us an interesting (non-AI related) fact about you?I have lived and studied in four different countries.

About Amar

| | Amar is a PhD student at the Institute of Artificial Intelligence of Ulm University in Germany. His research focuses on Explainable Artificial Intelligence (XAI) in Human-Robot Interaction (HRI), particularly how robots can generate context-sensitive explanations for navigation decisions. He combines symbolic planning and machine learning to build explainable robot systems that adapt to human preferences and different contexts. Before starting his PhD, he studied Electrical Engineering at the University of Sarajevo in Sarajevo, Bosnia and Herzegovina, and Computer Science at Mälardalen University in Västerås, Sweden. Outside academia, Amar enjoys travelling, photography, and exploring connections between technology and society. |

View Details

Claire chatted to Amy LaViers from the Robotics, Automation, and Dance Lab about the creative relationship between humans and machines.

Amy LaViers works at the intersection of robotics and dance. Her writing, choreography, and machine designs have been presented internationally at performing arts and engineering venues, including Merce Cunningham’s studios, the Performance Arcade, and universities like Berkeley and Brown. She is the founder of three startup companies, including AE Machines, which won “Product Design of the Year” at the 4th Rev Awards in Chicago. Amy runs the Robotics, Automation, and Dance (RAD) Lab, a non-profit for art-making, commercialization, education, outreach, and research.

View Details

Claire chatted to Nick Thompson from BOW about software that makes robots easier to program.

Nick Thompson is CEO of BOW and exited founder of One Beyond Ltd, an international software development firm. His career started in 1997 as a software engineer, founded One Beyond in the early 2000’s and after 20 years in the business sold to a private equity firm. In 2022 he was recognised as one of the UK’s ‘Most Ambitious Business Leaders’ by LDC Private Equity Group.

View Details

The AAMAS 2025 best paper and demo awards were presented at the 24th International Conference on Autonomous Agents and Multiagent Systems, which took place from 19-23 May 2025 in Detroit. The Distinguished Dissertation Award was also recently announced. The winners in the various categories are as follows:

Best Paper AwardWinner Soft Condorcet Optimization for Ranking of General Agents, Marc Lanctot, Kate Larson, Michael Kaisers, Quentin Berthet, Ian Gemp, Manfred Diaz, Roberto-Rafael Maura-Rivero, Yoram Bachrach, Anna Koop, Doina Precup*

Finalists Azorus: Commitments over Protocols for BDI Agents, Amit K. Chopra, Matteo Baldoni, Samuel H. Christie V, Munindar P. Singh * Curiosity-Driven Partner Selection Accelerates Convention Emergence in Language Games, Chin-Wing Leung, Paolo Turrini, Ann Nowe * Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards, Fangqi Liu, Rishav Sen, Jose Paolo Talusan, Ava Pettet, Aaron Kandel, Yoshinori Suzue, Ayan Mukhopadhyay, Abhishek Dubey * Ready, Bid, Go! On-Demand Delivery Using Fleets of Drones with Unknown, Heterogeneous Energy Storage Constraints, Mohamed S. Talamali, Genki Miyauchi, Thomas Watteyne, Micael Santos Couceiro, Roderich Gross*

Pragnesh Jay Modi Best Student Paper AwardWinners Decentralized Planning Using Probabilistic Hyperproperties, Francesco Pontiggia, Filip Macák, Roman Andriushchenko, Michele Chiari, Milan Ceska * Large Language Models for Virtual Human Gesture Selection, Parisa Ghanad Torshizi, Laura B. Hensel, Ari Shapiro, Stacy Marsella*

Runner-up ReSCOM: Reward-Shaped Curriculum for Efficient Multi-Agent Communication Learning, Xinghai Wei, Tingting Yuan, Jie Yuan, Dongxiao Liu, Xiaoming Fu*

Finalists Explaining Facial Expression Recognition, Sanjeev Nahulanthran, Leimin Tian, Dana Kulic, Mor Vered * Agent-Based Analysis of Green Disclosure Policies and Their Market-Wide Impact on Firm Behavior, Lingxiao Zhao, Maria Polukarov, Carmine Ventre*

Blue Sky Ideas Track Best Paper AwardWinner Grounding Agent Reasoning in Image Schemas: A Neurosymbolic Approach to Embodied Cognition, François Olivier, Zied Bouraoui*

Finalist Towards Foundation-model-based multiagent system to Accelerate AI for social impact, Yunfan Zhao, Niclas Boehmer, Aparna Taneja, Milind Tambe*

Best Demo AwardWinner Serious Games for Ethical Preference Elicitation, Jayati Deshmukh, Zijie Liang, Vahid Yazdanpanah, Sebastian Stein, Sarvapali Ramchurn*

Victor Lesser Distinguished Dissertation AwardThe Victor Lesser Distinguished Dissertation Award is given for dissertations in the field of autonomous agents and multiagent systems that show originality, depth, impact, as well as quality of writing, supported by high-quality publications.

Winner Jannik Peters. Thesis title: Facets of Proportionality: Selecting Committees, Budgets, and Clusters*

Runner-up Lily Xu. Thesis title: High-stakes decisions from low-quality data: AI decision-making for planetary health*

View Details

The 2025 IEEE International Conference on Robotics and Automation (ICRA) best paper winners and finalists in the various different categories have been announced. The recipients were revealed during an award ceremony at the conference, which took place from 19-23 May in Atlanta, USA.

IEEE ICRA Best Paper Award on Robot LearningWinner Robo-DM: Data Management for Large Robot Datasets, Kaiyuan Chen, Letian Fu, David Huang, Yanxiang Zhang, Yunliang Lawrence Chen, Huang Huang, Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag Sanketi, John Kubiatowicz, Ken Goldberg

Finalists Achieving Human Level Competitive Robot Table Tennis, David D’Ambrosio, Saminda Wishwajith Abeyruwan, Laura Graesser, Atil Iscen, Heni Ben Amor, Alex Bewley, Barney J. Reed, Krista Reymann, Leila Takayama, Yuval Tassa, Krysztof Choromanski, Erwin Coumans, Deepali Jain, Navdeep Jaitly, Natasha Jaques, Satoshi Kataoka, Yuheng Kuang, Nevena Lazic, Reza, Mahjourian, Sherry Moore, Kenneth Oslund, Anish Shankar, Vikas Sindhwani, Vincent Vanhoucke, Grace Vesom, Peng Xu, Pannag Sanketi * No Plan but Everything under Control: Robustly Solving Sequential Tasks with Dynamically Composed Gradient Descent, Vito Mengers, Oliver Brock

IEEE ICRA Best Paper Award in Field and Service RoboticsWinner PolyTouch: A Robust Multi-Modal Tactile Sensor for Contact-Rich Manipulation Using Tactile-Diffusion Policies, Jialiang Zhao, Naveen Kuppuswamy, Siyuan Feng, Benjamin Burchfiel, Edward Adelson

Finalists A New Stereo Fisheye Event Camera for Fast Drone Detection and Tracking, Daniel Rodrigues Da Costa, Maxime Robic, Pascal Vasseur, Fabio Morbidi * Learning-Based Adaptive Navigation for Scalar Field Mapping and Feature Tracking, Jose Fuentes, Paulo Padrao, Abdullah Al Redwan Newaz, Leonardo Bobadilla

IEEE ICRA Best Paper Award on Human-Robot InteractionWinner Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition, Shengchent Luo, Quanuan Peng, Jun Lv, Kaiwen Hong, Katherin Driggs-Campbell, Cewu Lu, Yong-Lu Li

Finalists To Ask or Not to Ask: Human-In-The-Loop Contextual Bandits with Applications in Robot-Assisted Feeding, Rohan Banerjee, Rajat Kumar Jenamani, Sidharth Vasudev, Amal Nanavati, Katherine Dimitropoulou, Sarah Dean, Tapomayukh Bhattacharjee * Point and Go: Intuitive Reference Frame Reallocation in Mode Switching for Assistive Robotics, Allie Wang, Chen Jiang, Michael Przystupa, Justin Valentine, Martin Jagersand*

IEEE ICRA Best Paper Award on Mechanisms and DesignWinner Individual and Collective Behaviors in Soft Robot Worms Inspired by Living Worm Blobs, Carina Kaeser, Junghan Kwon, Elio Challita, Harry Tuazon, Robert Wood, Saad Bhamla, Justin Werfel*

Finalists Informed Repurposing of Quadruped Legs for New Tasks, Fuchen Chen, Daniel Aukes * Intelligent Self-Healing Artificial Muscle: Mechanisms for Damage Detection and Autonomous, Ethan Krings, Patrick Mcmanigal, Eric Markvicka*

IEEE ICRA Best Paper Award on Planning and ControlWinner No Plan but Everything under Control: Robustly Solving Sequential Tasks with Dynamically Composed Gradient Descent, Vito Mengers, Oliver Brock

Finalists SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models, Yi Wu, Zikang Xiong, Yiran Hu, Shreyash Sridhar Iyengar, Nan Jiang, Aniket Bera, Lin Tan, Suresh Jagannathan * Marginalizing and Conditioning Gaussians Onto Linear Approximations of Smooth Manifolds with Applications in Robotics, Zi Cong Guo, James Richard Forbes, Timothy Barfoot*

IEEE ICRA Best Paper Award in Robot PerceptionWinner MAC-VO: Metrics-Aware Covariance for Learning-Based Stereo Visual Odometry, Yuheng Qju, Yutian Chen, Zihao Zhang, Wenshan Wang, Sebastian Scherer

Finalists Ground-Optimized 4D Radar-Inertial Odometry Via Continuous Velocity Integration Using Gaussian Process, Wooseong Yang, Hyesu Jang, Ayoung Kim * UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation, Yihe Tang, Wenlong Huang, Yingke Wang, Chengshu Li, Roy Yuan, Ruohan Zhang, Jiajun Wu, Li Fei-Fei*

IEEE ICRA Best Paper Award in Robot Manipulation and LocomotionWinner D(R, O) Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping, Zhenyu Wei, Zhixuan Xu, Jingxiang Guo, Yiwen Hou, Chongkai Gao, Zhehao Cai, Jiayu Luo, Lin Shao

Finalists Full-Order Sampling-Based MPC for Torque-Level Locomotion Control Via Diffusion-Style Annealing, Haoru Xue, Chaoyi Pan, Zeji Yi, Guannan Qu, Guanya Shi * TrofyBot: A Transformable Rolling and Flying Robot with High Energy Efficiency, Mingwei Lai, Yugian Ye, Hanyu Wu, Chice Xuan, Ruibin Zhang, Qiuyu Ren, Chao Xu, Fei Gao, Yanjun Cao*

IEEE ICRA Best Paper Award in AutomationWinner Physics-Aware Robotic Palletization with Online Masking Inference, Tiangi Zhang, Zheng Wu, Yuxin Chen, Yixiao Wang, Boyuan Liang, Scott Moura, Masayoshi Tomizuka, Mingyu Ding, Wei Zhan

Finalists In-Plane Manipulation of Soft Micro-Fiber with Ultrasonic Transducer Array and Microscope, Jieyun Zou, Siyuan An, Mingyue Wang, Jiaqi Li, Yalin Shi, You-Fu Li, Song Liu * A Complete and Bounded-Suboptimal Algorithm for a Moving Target Traveling Salesman Problem with Obstacles in 3D, Anoop Bhat, Geordan Gutow, Bhaskhar Vundurthy, Zhonggiang, Sivakumar Rathinam, Howie Choset*

IEEE ICRA Best Paper Award in Medical RoboticsWinner In-Vivo Tendon-Driven Rodent Ankle Exoskeleton System for Sensorimotor Rehabilitation, Juwan Han, Seunghyeon Park, Keehon Kim

Finalists Image-Based Compliance Control for Robotic Steering of a Ferromagnetic Guidewire, An Hu, Chen Sun, Adam Dmytriw, Nan Xiao, Yu Sun * AutoPeel: Adhesion-Aware Safe Peeling Trajectory Optimization for Robotic Wound Care, Xiao Liang, Youcheng Zhang, Fei Liu, Florian Richter, Michael C. Yip*

IEEE ICRA Best Paper Award on Multi-Robot SystemsWinner Deploying Ten Thousand Robots: Scalable Imitation Learning for Lifelong Multi-Agent Path Finding, He Jiang, Yutong Wang, Rishi Veerapaneni, Tanishq Harish Duhan, Guillaume Adrien Sartoretti, Jiaoyang Li

Finalists Distributed Multi-Robot Source Seeking in Unknown Environments with Unknown Number of Sources, Lingpeng Chen, Siva Kailas, Srujan Deolasee, Wenhao Luo, Katia Sycara, Woojun Kim * Multi-Nonholonomic Robot Object Transportation with Obstacle Crossing Using a Deformable Sheet, Weijian Zhang, Charlie Street, Masoumeh Mansouri

IEEE ICRA Best Conference Paper AwardWinners Marginalizing and Conditioning Gaussians Onto Linear Approximations of Smooth Manifolds with Applications in Robotics, Zi Cong Guo, James Richard Forbes, Timothy Barfoot * MAC-VO: Metrics-Aware Covariance for Learning-Based Stereo Visual Odometry, Yuheng Qju, Yutian Chen, Zihao Zhang, Wenshan Wang, Sebastian Scherer*

In addition to the papers listed above, these paper were also finalists for the IEEE ICRA Best Conference Paper Award.

Finalists MiniVLN: Efficient Vision-And-Language Navigation by Progressive Knowledge Distillation, Junyou Zhu, Yanyuan Qiao, Siqi Zhang, Xingjian He, Qi Wu, Jing Liu * RoboCrowd: Scaling Robot Data Collection through Crowdsourcing, Suvir Mirchandani, David D. Yuan, Kaylee Burns, Md Sazzad Islam, Zihao Zhao, Chelsea Finn, Dorsa Sadigh* * How Sound-Based Robot Communication Impacts Perceptions of Robotic Failure, Jai’La Lee Crider, Rhian Preston, Naomi T. Fitter* * Obstacle-Avoidant Leader Following with a Quadruped Robot, Carmen Scheidemann, Lennart Werner, Victor Reijgwart, Andrei Cramariuc, Joris Chomarat, Jia-Ruei Chiu, Roland Siegwart, Marco Hutter* * Dynamic Tube MPC: Learning Error Dynamics with Massively Parallel Simulation for Robust Safety in Practice, William Compton, Noel Csomay-Shanklin, Cole Johnson, Aaron Ames* * Bat-VUFN: Bat-Inspired Visual-And-Ultrasound Fusion Network for Robust Perception in Adverse Conditions, Gyeongrok Lim, Jeong-ui Hong, Min Hyeon Bae* * TinySense: A Lighter Weight and More Power-Efficient Avionics System for Flying Insect-Scale Robots, Zhitao Yu, Josh Tran, Claire Li, Aaron Weber, Yash P. Talwekar, Sawyer Fuller* * TSCLIP: Robust CLIP Fine-Tuning for Worldwide Cross-Regional Traffic Sign Recognition, Guoyang Zhao, Fulong Ma, Weiging Qi, Chenguang Zhang, Yuxuan Liu, Ming Liu, Jun Ma* * Geometric Design and Gait Co-Optimization for Soft Continuum Robots Swimming at Low and High Reynolds Numbers, Yanhao Yang, Ross Hatton* * ShadowTac: Dense Measurement of Shear and Normal Deformation of a Tactile Membrane from Colored Shadows, Giuseppe Vitrani, Basile Pasquale, Michael Wiertlewski* * Occlusion-aware 6D Pose Estimation with Depth-guided Graph Encoding and Cross-semantic Fusion for Robotic Grasping, Jingyang Liu, Zhenyu Lu, Lu Chen, Jing Yang, Chenguang Yang* * Stable Tracking of Eye Gaze Direction During Ophthalmic Surgery, Tinghe Hong, Shenlin Cai, Boyang Li, Kai Huang* * Configuration-Adaptive Visual Relative Localization for Spherical Modular Self-Reconfigurable Robots, Yuming Liu, Qiu Zheng, Yuxiao Tu, Yuan Gao, Guanqi Liang, Tin Lun Lam* * Realm: Real-Time Line-Of-Sight Maintenance in Multi-Robot Navigation with Unknown Obstacles, Ruofei Bai, Shenghai Yuan, Kun Li, Hongliang Guo, Wei-Yun Yau, Lihua Xie

IEEE ICRA Best Student Paper AwardWinners Deploying Ten Thousand Robots: Scalable Imitation Learning for Lifelong Multi-Agent Path Finding, He Jiang, Yutong Wang, Rishi Veerapaneni, Tanishq Harish Duhan, Guillaume Adrien Sartoretti, Jiaoyang Li * ShadowTac: Dense Measurement of Shear and Normal Deformation of a Tactile Membrane from Colored Shadows, Giuseppe Vitrani, Basile Pasquale, Michael Wiertlewski* * Point and Go: Intuitive Reference Frame Reallocation in Mode Switching for Assistive Robotics, Allie Wang, Chen Jiang, Michael Przystupa, Justin Valentine, Martin Jagersand * TinySense: A Lighter Weight and More Power-Efficient Avionics System for Flying Insect-Scale Robots, Zhitao Yu, Josh Tran, Claire Li, Aaron Weber, Yash P. Talwekar, Sawyer Fuller

Note: papers with an * were eligible for the IEEE ICRA Best Student Paper Award.

View Details

The 2025 IEEE International Conference on Robotics & Automation (ICRA) took place from 19–23 May, in Atlanta, USA. The event featured plenary and keynote sessions, tutorial and workshops, forums, and a community day. Find out what the participants got up during the conference.

Check out what’s happening at the #ICRA2025 Welcome Reception! pic.twitter.com/w66IQDFsku

— IEEE ICRA (@ieee_ras_icra) May 19, 2025

The excitement is real — #ICRA2025 is already buzzing! pic.twitter.com/DtVgLwiaTB

— IEEE ICRA (@ieee_ras_icra) May 19, 2025

ICRA #ICRA2025 #RoboticsInAfrica

[image or embed]

— Black in Robotics (@blackinrobotics.bsky.social) 18 May 2025 at 23:22

At #ICRA2025? Check out my student Yi Wu’s talk (TuCT1.4) at 3:30PM Tuesday in Room 302 at the Award Finalists 3 Session about how SELP Generates Safe and Efficient Plans for #Robot #Agents with #LLMs! #ConstrainedDecoding #LLMPlanner
@purduecs.bsky.social
@cerias.bsky.social

[image or embed]

— Lin Tan (@lin-tan.bsky.social) 19 May 2025 at 13:25

My MS student, Robel Mamo, is presenting his poster at #ICRA2025 #Field_Robotics workshop. His work is on "Crop-Aligned Cutout," a novel data augmentation method for under-canopy navigation @ieee_ras_icra @kennesawstate @KSUresearch pic.twitter.com/CBewagkpaQ

— Taeyeong Choi (최태영) (@ssuty) May 19, 2025

#ICRA2025 pic.twitter.com/FRfqgmSqNd

— Masato Kobayashi @るっと (@MeRTcooking) May 21, 2025

Malte Mosbach will present today 16:45 at #ICRA2025 in room 404 our paper:
"Prompt-responsive Object Retrieval with Memory-augmented Student-Teacher Learning"
www.ais.uni-bonn.de/videos/ICRA_…

[image or embed]

— Sven Behnke (@sven-behnke.bsky.social) 20 May 2025 at 15:57

#ICRA2025 pic.twitter.com/ANKoq3ry5K

— Masato Kobayashi @るっと (@MeRTcooking) May 21, 2025

Tomorrow morning at #ICRA2025, I will be presenting our findings on whether robots can learn dual-arm tasks from just a single demonstration. (Spoiler: they can!)

Come along!

This was led by my excellent PhD student Yilong Wang.

Paper & videos here: https://t.co/F2PtNJEJZT. pic.twitter.com/ycSAzuAgPn

— Edward Johns @ ICRA 2025 (@Ed__Johns) May 20, 2025

I will present our work on air-ground collaboration with SPOMP in 407A in a few minutes! We deployed 1 UAV and 3 UGVs in a fully autonomous mapping mission in large-scale environments. Come check it out! #ICRA2025 @grasplab.bsky.social

[image or embed]

— Fernando Cladera (@fcladera.bsky.social) 21 May 2025 at 20:13

Snapshots from #ICRA2025 @ieee_ras_icra : fans keep balloon walkers in constrained area @DennisHongRobot ; Artly coffee robot https://t.co/NCAMEo4fr5 ; mural ; and robo-friends #robots #artly #innovators pic.twitter.com/YAZdUWr9CD

— Heather Knight (@heatherknight) May 21, 2025

Cool things happening at #ICRA2025
RoboRacers gearing up for their qualifiers

[image or embed]

— Ameya Salvi (@ameyasalvi.bsky.social) 21 May 2025 at 13:56

Wednesday #ICRA2025 highlights included:

Plenary talk by Tessa Lau, CEO & Co-Founder, Dusty Robotics
Keynote & Technical Sessions
Community Day
ICRA Expo
Competitions
And more!

Check out tomorrow's events here: https://t.co/kS4WmAlZwJ pic.twitter.com/IboRH05KEI

— IEEE ICRA (@ieee_ras_icra) May 22, 2025

Our Community Building Day has been a success. LatinX in Robotics, Queer in Robotics, and Black in Robotics are just some of the incredible groups building community here at #ICRA2025! pic.twitter.com/EWNi0xY2NI

— IEEE ICRA (@ieee_ras_icra) May 21, 2025

New work at #ICRA2025!
Robust Robot Walker
We enable quadruped robots to pass tiny traps (bars, pits, poles) using only proprioception – no cameras, no depth!

Catch us at Thursday 16:55pm in Room 305!
https://t.co/571p4xTJ5c pic.twitter.com/03F2Gqf40D

— shaoting zhu (@ShaotingZ38103) May 22, 2025

Robot parade at @ieee_ras_icra #ICRA2025 pic.twitter.com/q2wHcQQN3R

— Sriram (@SriRam2528) May 22, 2025

Prof. Concha Monje presenting our BSc degree in Robotics Engineering at #ICRA2025 Forum on Undergraduate Robotics in Atlanta @uc3m @EPS_UC3M @ofic_eps_uc3m @ieeeras @ieee_ras_icra @mecanohumano https://t.co/R8fjbvJ6V3 pic.twitter.com/kDiWqSgx8M

— uc3mRoboticsLab (@uc3mRoboticsLab) May 22, 2025

We received the #ICRA2025 #HRI #Award for Arts & Robotics on our co-painting robot

This project shows how arts and robotics can be used as a testbed to create better robots and to discover new knowledge about humans.@ieee_ras_icra @UMRobotics @DARPA pic.twitter.com/T4VKxH2dKP

— patrícia alves-oliveira (@p_alvesoliveira) May 23, 2025

Fun times at the #ICRA2025 farewell reception! Celebrating Atlanta style with a block party. pic.twitter.com/rg71RPWWKc

— IEEE ICRA (@ieee_ras_icra) May 22, 2025

View Details

Claire chatted to Jane Pauline Ramos Ramirez from Delft University of Technology about drones that can move on land and in the air.

Jane Pauline Ramos Ramirez is a licensed engineer with a multidisciplinary background in bionics, mechanical, and aerospace engineering, and international research experience. Her life’s work is rooted in designing inclusive, socially accessible systems that work in synergy with nature and create meaningful impact in communities. As part of this mission, she has been developing nature-inspired drones that can move on both land and in the air — blending her appreciation for nature, design, and the mechanics of how things work.

View Details

Claire chatted to Lerrel Pinto from New York University about using machine learning to train robots to adapt to new environments.

Lerrel Pinto is an Assistant Professor of Computer Science at New York University (NYU). His research is aimed at getting robots to generalize and adapt in the messy world we live in. His lab focuses broadly on robot learning and decision making, with an emphasis on large-scale learning (both data and models); representation learning for sensory data; developing algorithms to model actions and behaviour; reinforcement learning for adapting to new scenarios; and building open-source, affordable robots.

View Details

The 2025 IEEE International Conference on Robotics and Automation (ICRA) will take place from 19-23 May, in Atlanta, USA. The event will feature plenary talks, technical sessions, posters, workshops and tutorials, forums, and a science communication short course.

Plenary speakersThere are three plenary sessions this year. The speakers are as follows:

• Allison Okamura (Stanford University) – Rewired: The Interplay of Robots and Society
• Tessa Lau (Dusty Robotics) – So you want to build a robot company?
• Raffaello (Raff) D’Andrea (ETH Zurich) – Models are dead, long live models!

Keynote sessionsTuesday 20, Wednesday 21 and Thursday 22 will see a total of 12 keynote sessions. The featured topics and speakers are:

• Rehabilitation & Physically Assistive Systems
• Brenna Argall
• Robert Gregg
• Keehoon Kim

• Christina Piazza

• Optimization & Control

• Todd Murphey
• Angela Schoellig
• Jana Tumova

• Ram Vasudevan

• Human Robot Interaction

• Sonia Chernova
• Dongheui Lee
• Harold Soh

• Holly Yanco

• Soft Robotics

• Robert Katzschmann
• Hugo Rodrigue
• Cynthia Sung

• Wenzhen Yuan

• Field Robotics

• Margarita Chli
• Tobias Fischer
• Joshua Mangelson

• Inna Sharf

• Bio-inspired Robotics

• Kyujin Cho
• Dario Floreano
• Talia Moore

• Yasemin Ozkan-Aydin

• Haptics

• Jeremy Brown
• Matej Hoffman
• Tania Morimoto

• Jee-Hwan Ryu

• Planning

• Hanna Kurniawati
• Jen Jen Chung
• Dan Halperin

• Jing Xiao

• Manipulation

• Tamim Asfour
• Yasuhisa Hasegawa
• Alberto Rodriguez

• Shuran Song

• Locomotion

• Sarah Bergbreiter
• Cosimo Della Santina
• Hae-Won Park

• Ludovic Righetti

• Safety & Formal Methods

• Chuchu Fan
• Meng Guo
• Changliu Liu

• Pian Yu

• Multi-robot Systems

• Sabine Hauert
• Dimitra Panagou
• Alyssa Pierson
• Fumin Zhang

Science communication trainingJoin Sabine Hauert, Evan Ackerman and Laura Bridgeman for a crash course on science communication. In this concise tutorial, you will learn how to share your work with a broader audience. This session will take place on 22 May, 11:00 – 12:15.

Workshops and tutorialsThe programme of workshops and tutorials will take place on Monday 19 May and Friday 23 May. There are 59 events to choose from, and you can see the full list here.

ForumsThere will be three forums as part of the programme, one each on Tuesday 20, Wednesday 21 and Thursday 22.

• Robot Ethics Forum
• Harnessing Learning, Data, Foundation Models and Open Source: How African Scientists are Advancing Robotics Research
• Undergraduate Robotics Education Programs: Structures, Platforms, and Approaches

Community building dayWednesday 21 May is community building day, with six events planned:

• Queer in Robotics: Building a Community and Generating Inclusive Guidelines
• Harnessing Learning, Data, Foundation Models and Open Source: How African Scientists are Advancing Robotics Research
• RAS-WiE Voices — Women shaping the future of robotics and automation
• Black in Robotics and Blacks in Technology Social
• LatinX in AI & Robotics
• Community Building Day Dinner

Other eventsYou can find out more about the other sessions and event at the links below:

• Arts in robotics
• Career fair
• Competitions
• Stage presentations
• Technical tours

View Details

Kushal Kedia (left) and Prithwish Dan (right) are members of the development team behind RHyME, a system that allows robots to learn tasks by watching a single how-to video.

By Louis DiPietro

Cornell researchers have developed a new robotic framework powered by artificial intelligence – called RHyME (Retrieval for Hybrid Imitation under Mismatched Execution) – that allows robots to learn tasks by watching a single how-to video. RHyME could fast-track the development and deployment of robotic systems by significantly reducing the time, energy and money needed to train them, the researchers said.

“One of the annoying things about working with robots is collecting so much data on the robot doing different tasks,” said Kushal Kedia, a doctoral student in the field of computer science and lead author of a corresponding paper on RHyME. “That’s not how humans do tasks. We look at other people as inspiration.”

Kedia will present the paper, One-Shot Imitation under Mismatched Execution, in May at the Institute of Electrical and Electronics Engineers’ International Conference on Robotics and Automation, in Atlanta.

Home robot assistants are still a long way off – it is a very difficult task to train robots to deal with all the potential scenarios that they could encounter in the real world. To get robots up to speed, researchers like Kedia are training them with what amounts to how-to videos – human demonstrations of various tasks in a lab setting. The hope with this approach, a branch of machine learning called “imitation learning,” is that robots will learn a sequence of tasks faster and be able to adapt to real-world environments.

“Our work is like translating French to English – we’re translating any given task from human to robot,” said senior author Sanjiban Choudhury, assistant professor of computer science in the Cornell Ann S. Bowers College of Computing and Information Science.

This translation task still faces a broader challenge, however: Humans move too fluidly for a robot to track and mimic, and training robots with video requires gobs of it. Further, video demonstrations – of, say, picking up a napkin or stacking dinner plates – must be performed slowly and flawlessly, since any mismatch in actions between the video and the robot has historically spelled doom for robot learning, the researchers said.

“If a human moves in a way that’s any different from how a robot moves, the method immediately falls apart,” Choudhury said. “Our thinking was, ‘Can we find a principled way to deal with this mismatch between how humans and robots do tasks?’”

RHyME is the team’s answer – a scalable approach that makes robots less finicky and more adaptive. It trains a robotic system to store previous examples in its memory bank and connect the dots when performing tasks it has viewed only once by drawing on videos it has seen. For example, a RHyME-equipped robot shown a video of a human fetching a mug from the counter and placing it in a nearby sink will comb its bank of videos and draw inspiration from similar actions – like grasping a cup and lowering a utensil.

RHyME paves the way for robots to learn multiple-step sequences while significantly lowering the amount of robot data needed for training, the researchers said. They claim that RHyME requires just 30 minutes of robot data; in a lab setting, robots trained using the system achieved a more than 50% increase in task success compared to previous methods.

“This work is a departure from how robots are programmed today. The status quo of programming robots is thousands of hours of tele-operation to teach the robot how to do tasks. That’s just impossible,” Choudhury said. “With RHyME, we’re moving away from that and learning to train robots in a more scalable way.”

This research was supported by Google, OpenAI, the U.S. Office of Naval Research and the National Science Foundation.

Read the work in fullOne-Shot Imitation under Mismatched Execution, Kushal Kedia, Prithwish Dan, Angela Chao, Maximus Adrian Pace, Sanjiban Choudhury.

View Details

Photo credit: Muntaka Chasant, reproduced under a CC BY-SA 4.0 license.

By Kaja Šeruga

Just outside the historic German town of Goslar, a sprawling industrial complex receives an endless stream of discarded electronics. On arrival, this electronic waste is laboriously prepared for recycling.

Electrocycling GmbH is one of the largest e-waste recycling facilities in Europe. Every year, it processes up to 80 000 tonnes of electronic waste, which comes in all shapes and forms.

Manual dismantlingDespite an impressive array of machinery, more than half of the site’s employees manually prepare the discarded items for recycling. They do this by sorting the incoming waste and removing batteries, which are a fire hazard and a major challenge in e-waste recycling.

“There are more and more devices, they are getting smaller, and they all contain lithium batteries, some of which are permanently installed, soldered or glued in place,” said Hannes Fröhlich, Electrocycling’s managing director.

“It’s not a dream job, dismantling these appliances every day with hammers and pliers. I think we can do better.”

Some of these tedious tasks could be performed by robots. However, the problem is that every time there is a change in the product or the process, the hardware and software need to be restructured. This can be costly and time-consuming.

To address this issue, an EU-funded research initiative named ReconCycle has managed to automate the process by creating robots that can reconfigure themselves for different tasks.

New territory for roboticsResearchers from Slovenia, Germany and Italy worked together on this issue at the Jožef Stefan Institute, Slovenia’s leading research facility, from 2020 to 2024.

The team developed adaptable AI-supported robots that are able to remove batteries from smoke detectors and radiator heat metres.

These two products can be found in most households and are replaced every five to eight years, creating large amounts of waste.

“The main challenge is that there are so many different versions of each device. Just think how many different remote controls there are,” said Dr Aleš Ude. He is head of the Department of Automatics, Biocybernetics and Robotics at the Jožef Stefan Institute and coordinates the ReconCycle research team.

In industrial settings, robots are usually programmed for one specific task, repeating exactly the same series of movements in a predictable environment.

Instead, the researchers set out to create a robot that can adapt to many different tasks, using state-of-the-art AI.

“We wanted to expand robotics, introduce robots where there aren’t any yet,” Ude said.

A growing problemWorking with Electrocycling, Ude’s international research team created an adaptable robotic work cell. This is a workspace that consists of at least one robot, its tools and equipment, and its controller.

The novelty here is that this closed system autonomously adapts itself to various tasks, with the help of complex AI-driven software and modular hardware that can be quickly reconfigured. It also uses soft components like SoftHand, a human-like hand that can manipulate objects with great precision.

There are also safety features like collaborative robots and emergency stop buttons.

International collaboration was crucial in securing the right expertise, said Ude.

“Robotics is very interdisciplinary, so it’s difficult to find the right partners in one country.”

Thankfully, the new robots are arriving just at the right time, as the amount of e-waste produced every year continues to grow. Almost 5 million tonnes of e-waste are produced in the EU each year, amounting to about 11 kilograms per person. Less than 40% of that is recycled, the European Parliament has warned.

Globally, around 62 million tonnes of e-waste were produced in 2022 alone, enough to fill 1.5 million 40-tonne trucks, according to UN data. Even more worryingly, the amount of e-waste is rising five times faster than the amount that is being recycled.

The EU is working to reduce e-waste through the Waste from Electrical and Electronic Equipment Directive, which sets the standards for collection and recycling.

The work of Ude’s team is also aligned with the EU’s digital strategy, which encourages the use of AI in manufacturing to improve efficiency and help achieve climate neutrality by 2050.

Throwing away moneyE-waste also has serious economic implications. An estimated €84 billion is lost each year when valuable metals like copper, iron and gold are discarded instead of being reused, according to the UN’s global e-waste monitor.

At Electrocycling, 80% of the e-waste is recovered as raw materials, such as iron, zinc, gold, silver and palladium – some 35 materials in all.

“People need to understand that this is not just waste, but also raw materials that need to be recycled and kept in circulation, both for economic efficiency and a reduction of CO2,” said Fröhlich.

New technology can make it even more efficient, and Fröhlich sees a lot of potential in it.

“I was surprised by how far the technology and AI have already come,” he said. “They even recreated a human hand for the robot.”

Ude hopes to continue working with Electrocycling to improve e-waste solutions further. The hope is also that adaptable robots which can handle changing environments will have applications far beyond e-waste recycling.

Given more time and development, these robots could even handle general housekeeping, or support carers in senior homes, said Ude.

“Robotics could be of great help in such areas.”

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

Claire chatted to Emma Hart from Edinburgh Napier University about algorithms that ‘evolve’ better robot designs and control systems.

Emma Hart is a computer scientist working in the field of evolutionary computation. Her work takes inspiration from the natural world, in particular biological evolution, and uses this to develop algorithms that ‘evolve’ both the design and control systems of a robot, customised to a specific application. She was elected as a Fellow of the Royal Society of Edinburgh in 2022, and was awarded the ACM SIGEVO Award for Outstanding Contribution to Evolutionary Computation in 2023. She was invited to give a TED Talk on her work in 2021 that has over 1.8 million views.

View Details

Claire chatted to Will Kinghorn from Made Smarter about how to increase adoption of new tech by small manufacturers.

Will Kinghorn is an automation and robotics specialist for the Made Smarter Adoption Programme in the UK. With a background as a chartered manufacturing engineer in the aerospace industry, Will has extensive experience in developing and implementing automation and robotic solutions. He now works with smaller manufacturing companies, assessing their needs, identifying suitable technologies, and guiding them through the adoption process. Last year he released a book called ‘Digital Transformation in Your Manufacturing Business – A Made Smarter Guide’.

View Details

By Kristýna Janovská and Pavel Surynek

Imagine if all of our cars could drive themselves – autonomous driving is becoming possible, but to what extent? To get a vehicle somewhere by itself may not seem so tricky if the route is clear and well defined, but what if there are more cars, each trying to get to a different place? And what if we add pedestrians, animals and other unaccounted for elements? This problem has recently been increasingly studied, and already used in scenarios such as warehouse logistics, where a group of robots move boxes in a warehouse, each with its own goal, but all moving while making sure not to collide and making their routes – paths – as short as possible. But how to formalize such a problem? The answer is MAPF – multi-agent path finding [Silver, 2005].

Multi-agent path finding describes a problem where we have a group of agents – robots, vehicles or even people – who are each trying to get from their starting positions to their goal positions all at once without ever colliding (being in the same position at the same time).

Typically, this problem has been solved on graphs. Graphs are structures that are able to simplify an environment using its focal points and interconnections between them. These points are called vertices and can represent, for example, coordinates. They are connected by edges, which connect neighbouring vertices and represent distances between them.

If however we are trying to solve a real-life scenario, we strive to get as close to simulating reality as possible. Therefore, discrete representation (using a finite number of vertices) may not suffice. But how to search an environment that is continuous, that is, one where there is basically an infinite amount of vertices connected by edges of infinitely small sizes?

This is where something called sampling-based algorithms comes into play. Algorithms such as RRT* [Karaman and Frazzoli, 2011], which we used in our work, randomly select (sample) coordinates in our coordinate space and use them as vertices. The more points that are sampled, the more accurate the representation of the environment is. These vertices are connected to that of their nearest neighbours which minimizes the length of the path from the starting point to the newly sampled point. The path is a sequence of vertices, measured as a sum of the lengths of edges between them.

Figure 1: Two examples of paths connecting starting positions (blue) and goal positions (green) of three agents. Once an obstacle is present, agents plan smooth curved paths around it, successfully avoiding both the obstacle and each other.

We can get a close to optimal path this way, though there is still one problem. Paths created this way are still somewhat bumpy, as the transition between different segments of a path is sharp. If a vehicle was to take this path, it would probably have to turn itself at once when it reaches the end of a segment, as some robotic vacuum cleaners do when moving around. This slows the vehicle or a robot down significantly. A way we can solve this is to take these paths and smooth them, so that the transitions are no longer sharp, but smooth curves. This way, robots or vehicles moving on them can smoothly travel without ever stopping or slowing down significantly when in need of a turn.

Our paper [Janovská and Surynek, 2024] proposed a method for multi-agent path finding in continuous environments, where agents move on sets of smooth paths without colliding. Our algorithm is inspired by the Conflict Based Search (CBS) [Sharon et al., 2014]. Our extension into a continuous space called Continuous-Environment Conflict-Based Search (CE-CBS) works on two levels:

Figure 2: Comparison of paths found with discrete CBS algorithm on a 2D grid (left) and CE-CBS paths in a continuous version of the same environment. Three agents move from blue starting points to green goal points. These experiments are performed in the Robotic Agents Laboratory at Faculty of Information Technology of the Czech Technical University in Prague.

Firstly, each agent searches for a path individually. This is done with the RRT* algorithm as mentioned above. The resulting path is then smoothed using B-spline curves, polynomial piecewise curves applied to vertices of the path. This removes sharp turns and makes the path easier to traverse for a physical agent.

Individual paths are then sent to the higher level of the algorithm, in which paths are compared and conflicts are found. Conflict arises if two agents (which are represented as rigid circular bodies) overlap at any given time. If so, constraints are created to forbid one of the agents from passing through the conflicting space at a time interval during which it was previously present in that space. Both options which constrain one of the agents are tried – a tree of possible constraint settings and their solutions is constructed and expanded upon with each conflict found. When a new constraint is added, this information passes to all agents it concerns and their paths are re-planned so that they avoid the constrained time and space. Then the paths are checked again for validity, and this repeats until a conflict-free solution, which aims to be as short as possible is found.

This way, agents can effectively move without losing speed while turning and without colliding with each other. Although there are environments such as narrow hallways where slowing down or even stopping may be necessary for agents to safely pass, CE-CBS finds solutions in most environments.

This research is supported by the Czech Science Foundation, 22-31346S.

You can read our paper here.

References* Janovská, K. and Surynek, P. (2024). Multi-agent Path Finding in Continuous Environment, CoRR. * Sharon, G., Stern, R., Felner, A., and Sturtevant, N. R. (2014). Conflict-based search for optimal multi-agent pathfinding, Artificial Intelligence. * Karaman, S. and Frazzoli, E. (2011). Sampling-based algorithms for optimal motion planning, CoRR. * Piegl, L. and Tiller, W. (1996). The NURBS Book, Springer-Verlag, New York, USA, second edition. * Silver, D. (2005). Cooperative pathfinding, Proceedings of the First Artificial Intelligence and Interactive Digital Entertainment Conference, Marina del Rey, California, USA.

View Details

Yuki Mitsufuji is a Lead Research Scientist at Sony AI. Yuki and his team presented two papers at the recent Conference on Neural Information Processing Systems (NeurIPS 2024). These works tackle different aspects of image generation and are entitled: GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping and PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher . We caught up with Yuki to find out more about this research.

There are two pieces of research we’d like to ask you about today. Could we start with the GenWarp paper? Could you outline the problem that you were focused on in this work?The problem we aimed to solve is called single-shot novel view synthesis, which is where you have one image and want to create another image of the same scene from a different camera angle. There has been a lot of work in this space, but a major challenge remains: when an image angle changes substantially, the image quality degrades significantly. We wanted to be able to generate a new image based on a single given image, as well as improve the quality, even in very challenging angle change settings.

How did you go about solving this problem – what was your methodology?The existing works in this space tend to take advantage of monocular depth estimation, which means only a single image is used to estimate depth. This depth information enables us to change the angle and change the image according to that angle – we call it “warp.” Of course, there will be some occluded parts in the image, and there will be information missing from the original image on how to create the image from a new angle. Therefore, there is always a second phase where another module can interpolate the occluded region. Because of these two phases, in the existing work in this area, geometrical errors introduced in warping cannot be compensated for in the interpolation phase.

We solve this problem by fusing everything together. We don’t go for a two-phase approach, but do it all at once in a single diffusion model. To preserve the semantic meaning of the image, we created another neural network that can extract the semantic information from a given image as well as monocular depth information. We inject it using a cross-attention mechanism, into the main base diffusion model. Since the warping and interpolation were done in one model, and the occluded part can be reconstructed very well together with the semantic information injected from outside, we saw the overall quality improved. We saw improvements in image quality both subjectively and objectively, using metrics such as FID and PSNR.

Can people see some of the images created using GenWarp?Yes, we actually have a demo, which consists of two parts. One shows the original image and the other shows the warped images from different angles.

Moving on to the PaGoDA paper, here you were addressing the high computational cost of diffusion models? How did you go about addressing that problem?Diffusion models are very popular, but it’s well-known that they are very costly for training and inference. We address this issue by proposing PaGoDA, our model which addresses both training efficiency and inference efficiency.

It’s easy to talk about inference efficiency, which directly connects to the speed of generation. Diffusion usually takes a lot of iterative steps towards the final generated output – our goal was to skip these steps so that we could quickly generate an image in just one step. People call it “one-step generation” or “one-step diffusion.” It doesn’t always have to be one step; it could be two or three steps, for example, “few-step diffusion”. Basically, the target is to solve the bottleneck of diffusion, which is a time-consuming, multi-step iterative generation method.

In diffusion models, generating an output is typically a slow process, requiring many iterative steps to produce the final result. A key trend in advancing these models is training a “student model” that distills knowledge from a pre-trained diffusion model. This allows for faster generation—sometimes producing an image in just one step. These are often referred to as distilled diffusion models. Distillation means that, given a teacher (a diffusion model), we use this information to train another one-step efficient model. We call it distillation because we can distill the information from the original model, which has vast knowledge about generating good images.

However, both classic diffusion models and their distilled counterparts are usually tied to a fixed image resolution. This means that if we want a higher-resolution distilled diffusion model capable of one-step generation, we would need to retrain the diffusion model and then distill it again at the desired resolution.

This makes the entire pipeline of training and generation quite tedious. Each time a higher resolution is needed, we have to retrain the diffusion model from scratch and go through the distillation process again, adding significant complexity and time to the workflow.

The uniqueness of PaGoDA is that we train across different resolution models in one system, which allows it to achieve one-step generation, making the workflow much more efficient.

For example, if we want to distill a model for images of 128×128, we can do that. But if we want to do it for another scale, 256×256 let’s say, then we should have the teacher train on 256×256. If we want to extend it even more for higher resolutions, then we need to do this multiple times. This can be very costly, so to avoid this, we use the idea of progressive growing training, which has already been studied in the area of generative adversarial networks (GANs), but not so much in the diffusion space. The idea is, given the teacher diffusion model trained on 64×64, we can distill information and train a one-step model for any resolution. For many resolution cases we can get a state-of-the-art performance using PaGoDA.

Could you give a rough idea of the difference in computational cost between your method and standard diffusion models. What kind of saving do you make?The idea is very simple – we just skip the iterative steps. It is highly dependent on the diffusion model you use, but a typical standard diffusion model in the past historically used about 1000 steps. And now, modern, well-optimized diffusion models require 79 steps. With our model that goes down to one step, we are looking at it about 80 times faster, in theory. Of course, it all depends on how you implement the system, and if there’s a parallelization mechanism on chips, people can exploit it.

Is there anything else you would like to add about either of the projects?Ultimately, we want to achieve real-time generation, and not just have this generation be limited to images. Real-time sound generation is an area that we are looking at.

Also, as you can see in the animation demo of GenWarp, the images change rapidly, making it look like an animation. However, the demo was created with many images generated with costly diffusion models offline. If we could achieve high-speed generation, let’s say with PaGoDA, then theoretically, we could create images from any angle on the fly.

Find out more: GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping, Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chieh-Hsin Lai, Seungryong Kim, Yuki Mitsufuji. * GenWarp demo * PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher, Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Yuhta Takida, Naoki Murata, Toshimitsu Uesaka, Yuki Mitsufuji, Stefano Ermon.*

About Yuki Mitsufuji

| | Yuki Mitsufuji is a Lead Research Scientist at Sony AI. In addition to his role at Sony AI, he is a Distinguished Engineer for Sony Group Corporation and the Head of Creative AI Lab for Sony R&D. Yuki holds a PhD in Information Science & Technology from the University of Tokyo. His groundbreaking work has made him a pioneer in foundational music and sound work, such as sound separation and other generative models that can be applied to music, sound, and other modalities. |

View Details

Claire chatted to Miranda Lowther from the University of Bristol about soft, sensitive electronic skin for prosthetic limbs.

Miranda Lowther is a PhD researcher at the FARSCOPE-TU Centre for Doctoral Training, a joint venture between University of Bristol, University of West of England, and Bristol Robotics Laboratory, where she is pursuing her passion for using soft robotics and morphological computation to help people in healthcare. For her PhD, she is investigating how soft e-skins and morphological computation concepts can be used to improve prosthetic user health, comfort, and quality of life, through sensing and adaptation.

View Details

In a series of interviews, we’re meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. In this latest interview, we hear from Amina Mević who is applying machine learning to semiconductor manufacturing. Find out more about her PhD research so far, what makes this field so interesting, and how she found the AAAI Doctoral Consortium experience.

Tell us a bit about your PhD – where are you studying, and what is the topic of your research?I am currently pursuing my PhD at the University of Sarajevo, Faculty of Electrical Engineering, Department of Computer Science and Informatics. My research is being carried out in collaboration with Infineon Technologies Austria as part of the Important Project of Common European Interest (IPCEI) in Microelectronics. The topic of my research focuses on developing an explainable multi-output virtual metrology system based on machine learning to predict the physical properties of metal layers in semiconductor manufacturing.

Could you give us an overview of the research you’ve carried out so far during your PhD?In the first year of my PhD, I worked on preprocessing complex manufacturing data and preparing a robust multi-output prediction setup for virtual metrology. I collaborated with industry experts to understand the process intricacies and validate the prediction models. I applied a projection-based selection algorithm (ProjSe), which aligned well with both domain knowledge and process physics.

In the second year, I developed an explanatory method, designed to identify the most relevant input features for multi-output predictions.

Is there an aspect of your research that has been particularly interesting?For me, the most interesting aspect is the synergy between physics, mathematics, cutting-edge technology, psychology, and ethics. I’m working with data collected during a physical process—physical vapor deposition—using concepts from geometry and algebra, particularly projection operators and their algebra, which have roots in quantum mechanics, to enhance both the performance and interpretability of machine learning models. Collaborating closely with engineers in the semiconductor industry has also been eye-opening, especially seeing how explanations can directly support human decision-making in high-stakes environments. I feel truly honored to deepen my knowledge across these fields and to conduct this multidisciplinary research.

What are your plans for building on your research so far during the PhD – what aspects will you be investigating next?I plan to focus more on time series data and develop explanatory methods for multivariate time series models. Additionally, I intend to investigate aspects of responsible AI within the semiconductor industry and ensure that the solutions proposed during my PhD align with the principles outlined in the EU AI Act.

How was the AAAI Doctoral Consortium, and the AAAI conference experience in general?Attending the AAAI Doctoral Consortium was an amazing experience! It gave me the opportunity to present my research and receive valuable feedback from leading AI researchers. The networking aspect was equally rewarding—I had inspiring conversations with fellow PhD students and mentors from around the world. The main conference itself was energizing and diverse, with cutting-edge research presented across so many AI subfields. It definitely strengthened my motivation and gave me new ideas for the final phase of my PhD.

Amina presenting two posters at AAAI 2025.

What made you want to study AI?After graduating in theoretical physics, I found that job opportunities—especially in physics research—were quite limited in my country. I began looking for roles where I could apply the mathematical knowledge and problem-solving skills I had developed during my studies. At the time, data science appeared to be an ideal and promising field. However, I soon realized that I missed the depth and purpose of fundamental research, which was often lacking in industry roles. That motivated me to pursue a PhD in AI, aiming to gain a deep, foundational understanding of the technology—one that can be applied meaningfully and used in service of humanity.

What advice would you give to someone thinking of doing a PhD in the field?Stay curious and open to learning from different disciplines—especially mathematics, statistics, and domain knowledge. Make sure your research has a purpose that resonates with you personally, as that passion will help carry you through challenges. There will be moments when you’ll feel like giving up, but before making any decision, ask yourself: am I just tired? Sometimes, rest is the solution to many of our problems. Finally, find mentors and communities to share ideas with and stay inspired.

Could you tell us an interesting (non-AI related) fact about you?I’m a huge science outreach enthusiast! I regularly volunteer with the Association for the Advancement of Science and Technology in Bosnia, where we run workshops and events to inspire kids and high school students to explore STEM—especially in underserved communities.

About Amina

| | Amina Mević is a PhD candidate and teaching assistant at the University of Sarajevo, Faculty of Electrical Engineering, Bosnia and Herzegovina. Her research is conducted in collaboration with Infineon Technologies Austria as part of the IPCEI in Microelectronics. She earned a master’s degree in theoretical physics and was awarded two Golden Badges of the University of Sarajevo for achieving a GPA higher than 9.5/10 during both her bachelor’s and master’s studies. Amina actively volunteers to promote STEM education among youth in Bosnia and Herzegovina and is dedicated to improving the research environment in her country. |

View Details

Claire chatted to Jeremy Hadall from the Satellite Applications Catapult about robotic systems for in-orbit servicing, assembly, and manufacturing.

Jeremy Hadall has worked with robotics for his entire career, developing novel and innovative approaches for manufacturing and logistics industries. He’s now turned his experience into the development of robots that enable those tasks in the orbital environment. Prior to joining the Satellite Applications Catapult, he served as Chief Engineer for Intelligent Automation at the Manufacturing Technology Centre for over ten years. He has previously served as a Royal Academy of Engineering Visiting Professor at Cranfield University.

View Details

Claire chatted to Tanja Katharina Kaiser from the University of Technology Nuremberg about how applying evolutionary principles can help robot teams make better decisions.

Tanja Katharina Kaiser is a senior researcher heading the Multi-Robot Systems Satellite Lab at the University of Technology Nuremberg (UTN) in Germany. She and her team focus on the development of adaptive multi-robot systems to solve complex real-world tasks using artificial intelligence. Tanja received her doctorate in robotics from the University of Lübeck in Germany in 2022. Before joining UTN, she held postdoctoral research positions at the Technical University of Dresden and the University of Konstanz.

View Details

Nadia Piet & Archival Images of AI + AIxDESIGN / Model Collapse / Licenced by CC-BY 4.0

By Jon Whittle, CSIRO and Stefan Harrer, CSIRO

In February this year, Google announced it was launching “a new AI system for scientists”. It said this system was a collaborative tool designed to help scientists “in creating novel hypotheses and research plans”.

It’s too early to tell just how useful this particular tool will be to scientists. But what is clear is that artificial intelligence (AI) more generally is already transforming science.

Last year for example, computer scientists won the Nobel Prize for Chemistry for developing an AI model to predict the shape of every protein known to mankind. Chair of the Nobel Committee, Heiner Linke, described the AI system as the achievement of a “50-year-old dream” that solved a notoriously difficult problem eluding scientists since the 1970s.

But while AI is allowing scientists to make technological breakthroughs that are otherwise decades away or out of reach entirely, there’s also a darker side to the use of AI in science: scientific misconduct is on the rise.

AI makes it easy to fabricate researchAcademic papers can be retracted if their data or findings are found to no longer valid. This can happen because of data fabrication, plagiarism or human error.

Paper retractions are increasing exponentially, passing 10,000 in 2023. These retracted papers were cited over 35,000 times.

One study found 8% of Dutch scientists admitted to serious research fraud, double the rate previously reported. Biomedical paper retractions have quadrupled in the past 20 years, the majority due to misconduct.

AI has the potential to make this problem even worse.

For example, the availability and increasing capability of generative AI programs such as ChatGPT makes it easy to fabricate research.

This was clearly demonstrated by two researchers who used AI to generate 288 complete fake academic finance papers predicting stock returns.

While this was an experiment to show what’s possible, it’s not hard to imagine how the technology could be used to generate fictitious clinical trial data, modify gene editing experimental data to conceal adverse results or for other malicious purposes.

Fake references and fabricated dataThere are already many reported cases of AI-generated papers passing peer-review and reaching publication – only to be retracted later on the grounds of undisclosed use of AI, some including serious flaws such as fake references and purposely fabricated data.

Some researchers are also using AI to review their peers’ work. Peer review of scientific papers is one of the fundamentals of scientific integrity. But it’s also incredibly time-consuming, with some scientists devoting hundreds of hours a year of unpaid labour. A Stanford-led study found that up to 17% of peer reviews for top AI conferences were written at least in part by AI.

In the extreme case, AI may end up writing research papers, which are then reviewed by another AI.

This risk is worsening the already problematic trend of an exponential increase in scientific publishing, while the average amount of genuinely new and interesting material in each paper has been declining.

AI can also lead to unintentional fabrication of scientific results.

A well-known problem of generative AI systems is when they make up an answer rather than saying they don’t know. This is known as “hallucination”.

We don’t know the extent to which AI hallucinations end up as errors in scientific papers. But a recent study on computer programming found that 52% of AI-generated answers to coding questions contained errors, and human oversight failed to correct them 39% of the time.

Maximising the benefits, minimising the risksDespite these worrying developments, we shouldn’t get carried away and discourage or even chastise the use of AI by scientists.

AI offers significant benefits to science. Researchers have used specialised AI models to solve scientific problems for many years. And generative AI models such as ChatGPT offer the promise of general-purpose AI scientific assistants that can carry out a range of tasks, working collaboratively with the scientist.

These AI models can be powerful lab assistants. For example, researchers at CSIRO are already developing AI lab robots that scientists can speak with and instruct like a human assistant to automate repetitive tasks.

A disruptive new technology will always have benefits and drawbacks. The challenge of the science community is to put appropriate policies and guardrails in place to ensure we maximise the benefits and minimise the risks.

AI’s potential to change the world of science and to help science make the world a better place is already proven. We now have a choice.

Do we embrace AI by advocating for and developing an AI code of conduct that enforces ethical and responsible use of AI in science? Or do we take a backseat and let a relatively small number of rogue actors discredit our fields and make us miss the opportunity?

Jon Whittle, Director, Data61, CSIRO and Stefan Harrer, Director, AI for Science, CSIRO

This article is republished from The Conversation under a Creative Commons license. Read the original article.

View Details

Claire chatted to Benjamin Mottis from ANYbotics about deploying their four-legged ANYmal robot in a variety of industries.

Benjamin Mottis is a Robotics Engineer in charge of ANYmal Research at ANYbotics. After graduating in robotics from EPFL, he joined ANYbotics as a Field Engineer in 2023. He specializes in deploying ANYmal and training customers across all ANYbotics verticals (Oil & Gas, Nuclear, Metals, Chemicals, etc.). Since 2024, as the Global Research Community Manager, he has been working on expanding the ANYmal Research Community and helping world-leading researchers push the boundaries of robotics with ANYmal.

View Details

Claire chatted to Josie Gotz from the Manufacturing Technology Centre about robotics for material recovery, reuse and recycling.

Josie Gotz is a Senior Research Engineer in the Intelligent Robotics Team at the Manufacturing Technology Centre. She works as the technical lead on a variety of robotics and automation projects from research and development through to integration across a wide variety of manufacturing sectors. She specialises in creating innovative solutions for these industries, combining vision systems and artificial intelligence to build flexible automation systems. Josie has a particular interest in automated disassembly for material recovery, reuse and recycling.

View Details

Congratulations to Shlomo Zilberstein on winning the 2025 ACM/SIGAI Autonomous Agents Research Award. This prestigious award is made for excellence in research in the area of autonomous agents. It is intended to recognize researchers in autonomous agents whose current work is an important influence on the field.

Professor Shlomo Zilberstein was recognised for his work establishing the field of decentralized Markov Decision Processes (DEC-MDPs), laying the groundwork for decision-theoretic planning in multi-agent systems and multi-agent reinforcement learning (MARL). The selection committee noted that these contributions have become a cornerstone of multi-agent decision-making, influencing researchers and practitioners alike.

Shlomo Zilberstein is Professor of Computer Science and former Associate Dean of Research at the University of Massachusetts Amherst. He is a Fellow of AAAI and the ACM, and has received numerous awards, including the UMass Chancellor’s Medal, the IFAAMAS Influential Paper Award, and the AAAI Distinguished Service Award.

View Details

Claire chatted to Kaspar Althoefer from Queen Mary University of London about soft robotic manipulators for healthcare and manufacturing.

Kaspar Althoefer is Director of the Centre for Advanced Robotics at Queen Mary University of London (QMUL). His research focuses on soft robotics, tactile perception, intelligent manipulation, and machine learning techniques for sensor signal interpretation. His research advancements have significant applications in robot-assisted minimally invasive surgery, rehabilitation, assistive technologies, and human-robot interactions within a range of scenarios, including manufacturing. Before joining QMUL, he was a Professor at King’s College London, where he also earned his PhD.

View Details

An original photograph taken by Felice Frankel (left) and an AI-generated image of the same content. Credit: Felice Frankel. Image on right was generated with DALL-E

By Melanie M Kaufman

For over 30 years, science photographer Felice Frankel has helped MIT professors, researchers, and students communicate their work visually. Throughout that time, she has seen the development of various tools to support the creation of compelling images: some helpful, and some antithetical to the effort of producing a trustworthy and complete representation of the research. In a recent opinion piece published in Nature magazine, Frankel discusses the burgeoning use of generative artificial intelligence (GenAI) in images and the challenges and implications it has for communicating research. On a more personal note, she questions whether there will still be a place for a science photographer in the research community.

Q: You’ve mentioned that as soon as a photo is taken, the image can be considered “manipulated.” There are ways you’ve manipulated your own images to create a visual that more successfully communicates the desired message. Where is the line between acceptable and unacceptable manipulation?

A: In the broadest sense, the decisions made on how to frame and structure the content of an image, along with which tools used to create the image, are already a manipulation of reality. We need to remember the image is merely a representation of the thing, and not the thing itself. Decisions have to be made when creating the image. The critical issue is not to manipulate the data, and in the case of most images, the data is the structure. For example, for an image I made some time ago, I digitally deleted the petri dish in which a yeast colony was growing, to bring attention to the stunning morphology of the colony. The data in the image is the morphology of the colony. I did not manipulate that data. However, I always indicate in the text if I have done something to an image. I discuss the idea of image enhancement in my handbook, “The Visual Elements, Photography”.

An image of a growing yeast colony where the petri dish has been digitally deleted. This type of manipulation could be acceptable because the actual data has not been manipulated, Frankel says. Image credit: Felice Frankel

Q: What can researchers do to make sure their research is communicated correctly and ethically?

A: With the advent of AI, I see three main issues concerning visual representation: the difference between illustration and documentation, the ethics around digital manipulation, and a continuing need for researchers to be trained in visual communication. For years, I have been trying to develop a visual literacy program for the present and upcoming classes of science and engineering researchers. MIT has a communication requirement which mostly addresses writing, but what about the visual, which is no longer tangential to a journal submission? I will bet that most readers of scientific articles go right to the figures, after they read the abstract.

We need to require students to learn how to critically look at a published graph or image and decide if there is something weird going on with it. We need to discuss the ethics of “nudging” an image to look a certain predetermined way. I describe in the article an incident when a student altered one of my images (without asking me) to match what the student wanted to visually communicate. I didn’t permit it, of course, and was disappointed that the ethics of such an alteration were not considered. We need to develop, at the very least, conversations on campus and, even better, create a visual literacy requirement along with the writing requirement.

Q: Generative AI is not going away. What do you see as the future for communicating science visually?

A: For the Nature article, I decided that a powerful way to question the use of AI in generating images was by example. I used one of the diffusion models to create an image using the following prompt:

“Create a photo of Moungi Bawendi’s nano crystals in vials against a black background, fluorescing at different wavelengths, depending on their size, when excited with UV light.”

The results of my AI experimentation were often cartoon-like images that could hardly pass as reality — let alone documentation — but there will be a time when they will be. In conversations with colleagues in research and computer-science communities, all agree that we should have clear standards on what is and is not allowed. And most importantly, a GenAI visual should never be allowed as documentation.

But AI-generated visuals will, in fact, be useful for illustration purposes. If an AI-generated visual is to be submitted to a journal (or, for that matter, be shown in a presentation), I believe the researcher MUST:

• clearly label if an image was created by an AI model;
• indicate what model was used;
• include what prompt was used; and
• include the image, if there is one, that was used to help the prompt.

View Details

Claire chatted to Vali Lalioti from the University of the Arts London about how art, culture and robotics interact.

Vali Lalioti is a pioneering designer, computer scientist and innovator. She has a PhD in Computer Science, an MRes in Design and an MBA, and extensive international leadership, research and innovation experience in Silicon Valley, Africa, China, Japan and Europe. Vali is passionate about how technology interacts with society and talks globally on women in tech, art and technology education and her research in societal applications for well-being, healthy ageing and art. She developed the first ever BBC Augmented Reality production in 2003 and has introduced the UK’s first Creative Robotics University Degrees.

View Details

Image taken from the front cover of the Future of AI Research report.

The Association for the Advancement of Artificial Intelligence (AAAI), has published a report on the Future of AI Research. The report, which was announced by outgoing AAAI President Francesca Rossi during the AAAI 2025 conference, covers 17 different AI topics and aims to clearly identify the trajectory of AI research in a structured way.

The report is the result of a Presidential Panel, chaired by Francesca Rossi, and comprising of 24 experienced AI researchers, who worked on the project between summer 2024 and spring 2025. As well as the views of the panel members, the report also draws on community feedback, which was received from 475 AI researchers via a survey.

The 17 topics, each with a dedicated chapter, are as follows.

• AI Reasoning
• AI Factuality & Trustworthiness
• AI Agents
• AI Evaluation
• AI Ethics & Safety
• Embodied AI
• AI & Cognitive Science
• Hardware & AI
• AI for Social Good
• AI & Sustainability
• AI for Scientific Discovery
• Artificial General Intelligence (AGI)
• AI Perception vs. Reality
• Diversity of AI Research Approaches
• Research Beyond the AI Research Community
• Role of Academia
• Geopolitical Aspects & Implications of AI

Each chapter includes a list of main takeaways, context and history, current state and trends, research challenges, and community opinion. You can read the report in full here.

View Details

Andrew Barto and Richard Sutton. Image credit: Association for Computing Machinery.

The Association for Computing Machinery, has named Andrew Barto and Richard Sutton as the recipients of the 2024 ACM A.M. Turing Award. The pair have received the honour for “developing the conceptual and algorithmic foundations of reinforcement learning”. In a series of papers beginning in the 1980s, Barto and Sutton introduced the main ideas, constructed the mathematical foundations, and developed important algorithms for reinforcement learning.

The Turing Award comes with a $1 million prize, to be split between the recipients. Since its inception in 1966, the award has honoured computer scientists and engineers on a yearly basis. The prize was last given for AI research in 2018, when Yoshua Bengio, Yann LeCun and Geoffrey Hinton were recognised for their contribution to the field of deep neural networks.

Andrew Barto is Professor Emeritus, Department of Information and Computer Sciences, University of Massachusetts, Amherst. He began his career at UMass Amherst as a postdoctoral Research Associate in 1977, and has subsequently held various positions including Associate Professor, Professor, and Department Chair. Barto received a BS degree in Mathematics (with distinction) from the University of Michigan, where he also earned his MS and PhD degrees in Computer and Communication Sciences.

Richard Sutton is a Professor in Computing Science at the University of Alberta, a Research Scientist at Keen Technologies (an artificial general intelligence company based in Dallas, Texas) and Chief Scientific Advisor of the Alberta Machine Intelligence Institute (Amii). Sutton was a Distinguished Research Scientist at Deep Mind from 2017 to 2023. Prior to joining the University of Alberta, he served as a Principal Technical Staff Member in the Artificial Intelligence Department at the AT&T Shannon Laboratory in Florham Park, New Jersey, from 1998 to 2002. Sutton received his BA in Psychology from Stanford University and earned his MS and PhD degrees in Computer and Information Science from the University of Massachusetts at Amherst.

The two researchers began collaborating in 1978, at the University of Massachusetts at Amherst, where Barto was Sutton’s PhD and postdoctoral advisor.

Find out more* ACM announcement * Richard Sutton in conversation with Stephen Hanson

View Details

Claire chatted to Patrick Meier from the Climate Robotics Network about how robots can help scale action on climate change.

Patrick Meier has 15+ years of leadership and field experience at the intersection of emerging tech, innovation, and social impact. He founded the Climate Robotics Network and currently leads the UK FCDO project on Robotics for Global Development in low- and middle-income countries. Previously, he served as Strategy Lead for Robotics at the Swiss Institute of Technology (EPFL) and Innovation Booster Robotics. He also co-founded and led WeRobotics, an international technology nonprofit with labs in 40+ countries.

View Details

Claire chatted to Catherine Menon from the University of Hertfordshire about designing home assistance robots with ethics in mind.

Catherine Menon is a principal lecturer at the University of Hertfordshire. Her research explores the ethics and safety of autonomous systems, and she has a particular interest in the interaction between safety requirements, ethical imperatives and trust constraints in public-facing AI including assistive robots. She has previously worked as a safety-critical systems engineer in the defence and nuclear sectors, and has been involved in producing and validating several international standards for these domains.

View Details

Claire chatted to Dan Nicholson from Maker Forge about creating open source robotics projects you can do at home.

Dan Nicholson is a seasoned Software Engineering Manager with over 20 years of experience as a software engineer and architect. Four years ago, he began exploring robotics as a hobby, which quickly evolved into a large-scale bipedal robotics project that has inspired a wide audience. After making the project open-source and 3D printable, Dan built a vibrant community around it, with over 25k followers. Through his platform, MakerForge.tech, Dan shares insights and project details while collaborating with partners and fellow makers to continue expanding the project’s impact.

View Details

Claire chatted to Anuradha Ranasinghe from Liverpool Hope University about haptic (touch) sensors for wearable tech and robotics.

Anuradha Ranasinghe earned her PhD in robotics from King’s College London in 2015, focusing on haptic-based human control in low-visibility conditions. She is now a senior lecturer in robotics at Liverpool Hope University, researching haptics, miniaturized sensors, and perception. Her work has received national and international media attention, including features by EPSRC, CBS Radio, Liverpool Echo, and Techxplore. She has published in leading robotics conferences and journals, and she has presented her findings at various international conferences.

View Details

The AAAI Award for Artificial Intelligence for the Benefit of Humanity recognizes positive impacts of artificial intelligence to protect, enhance, and improve human life in meaningful ways with long-lived effects. The award is given annually at the conference for the Association for the Advancement of Artificial Intelligence (AAAI).

This year, the AAAI Awards Committee has announced that the 2025 recipient of the award and $25,000 prize is Stuart J. Russell, “for his work on the conceptual and theoretical foundations of provably beneficial AI and his leadership in creating the field of AI safety”.

Stuart will give an invited talk at AAAI 2025 entitled “Can AI Benefit Humanity?”

About StuartStuart J. Russell is a Distinguished Professor of Computer Science at the University of California, Berkeley, and holds the Michael H. Smith and Lotfi A. Zadeh Chair in Engineering. He is also a Distinguished Professor of Computational Precision Health at UCSF. His research covers a wide range of topics in artificial intelligence including machine learning, probabilistic reasoning, knowledge representation, planning, real-time decision making, multitarget tracking, computer vision, computational physiology, and philosophical foundations. He has also worked with the United Nations to create a new global seismic monitoring system for the Comprehensive Nuclear-Test-Ban Treaty. His current concerns include the threat of autonomous weapons and the long-term future of artificial intelligence and its relation to humanity.

Read our content featuring previous winners of the award* 2024 award #AAAI2024 invited talk: Milind Tambe – using ML for social good * 2023 award: #AAAI2023 invited talk: Tuomas Sandholm on organ exchanges * 2022 award: Cynthia Rudin wins AAAI Squirrel AI Award * 2022 award: #AAAI2022 invited talk – Cynthia Rudin on interpretable machine learning * 2021 award: Regina Barzilay wins $1M Association for the Advancement of Artificial Intelligence Squirrel AI award * 2021 award: #AAAI2021 invited talk – Regina Barzilay on deploying machine learning methods in cancer diagnosis and drug design

View Details

Claire chatted to Robert Siddall from the University of Surrey about novel robot designs inspired by the way real animals move.

Robert Siddall is an aerospace engineer with an enthusiasm for unconventional robotics. He is interested in understanding animal locomotion for the benefit of synthetic locomotion, particularly flight. Before becoming a Lecturer at the University of Surrey, he worked at the Max Planck Institute for Intelligent Systems in Stuttgart, Germany, where he studied the arboreal acrobatics of rainforest-dwelling reptiles. His work focuses on the design of novel robots that can tackle important environmental problems.

View Details

Claire chatted to Didem Gurdur Broo from Uppsala University about how to shape the future of robotics, autonomous vehicles, and industrial automation.

Didem Gurdur Broo is an Assistant Professor and Associate Senior Lecturer at the Department of Information Technology at Uppsala University. She leads the Cyber-physical Systems Lab, directing research on intelligent systems like collaborative robots, autonomous vehicles, and smart cities. Didem is a computer scientist with a PhD in mechatronics, which can give you an idea about how much she loves to talk about the future and emerging technologies. She dreams a better world and actively works on improving inequalities regardless of their nature.

View Details

Claire chatted to Gianmarco Pisanelli from the the University of Sheffield Advanced Manufacturing Research Centre about how to promote the safe and intuitive use of robots in manufacturing.

Gianmarco Pisanelli specialises in early-stage technology readiness level (TRL) research, with a focus on collaborative workspaces, multi-robot systems, and robotic simulations. He possesses extensive experience in industrial robotics, including PLC programming and the development of intuitive robotic interfaces such as natural speech and augmented reality. As the lead developer for the Robot Operating System (ROS) at the University of Sheffield’s Advanced Manufacturing Research Centre (AMRC), he serves as the primary point of contact for SMEs and major partners.

View Details

Claire chatted to Kirstin Petersen from Cornell University about how robots can work together to achieve complex behaviours.

Kirstin Petersen is an Associate Professor in the School of Electrical and Computer Engineering at Cornell University. Her lab, the Collective Embodied Intelligence Lab, is focused on design and coordination of robot collectives able to achieve complex behaviors beyond the reach of an individual, and corresponding studies on how social insects do so in nature. Petersen did her postdoc at the Max Planck Institute for Intelligent Systems and her PhD at Harvard University and the Wyss Institute for Biologically Inspired Engineering.

View Details

Claire chatted to Keenan Wyrobek from Zipline about drones for delivering life-saving medicine to remote locations.

Keenan Wyrobek is co-founder and head of product and engineering at Zipline, the world’s first drone delivery service whose focus is delivering life-saving medicine to the most difficult to reach places on earth. Prior to Zipline, Keenan was a co-founder and director of the Personal Robotics Program at Willow Garage. He was involved in launching the Robot Operating System (ROS) and shipping PR2, the first personal robot for software R&D. Keenan has spent years delivering high tech products to market across a range of fields including consumer electronics and medical robotics.

View Details

Claire chatted to Isabella Fiorello from the University of Freiburg about plant-inspired robots made from living materials.

Isabella Fiorello is a Junior Group Leader and Principal Investigator of the Bioinspired Plant-hybrid Materials group at the University of Freiburg in Germany. She has a Master’s Degree in Industrial Biotechnology from the University of Turin in Italy and a PhD in Biorobotics from Scuola Superiore Sant’Anna in Italy. Her research focusses on the development of biologically-inspired microfabricated living materials able to precisely interact with complex unstructured surfaces for applications in precision agriculture, smart fabrics, space, and soft robotics.

View Details

Claire chatted to Christos Bergeles from King’s College London about micro-surgical robots to deliver therapies deep inside the body.

Christos Bergeles received his PhD in Robotics from ETH Zurich in Switzerland in 2011. As a Professor at King’s College London, he directs the “Robotics and Vision in Medicine Lab” whose mission is to develop micro-surgical robots that deliver regenerative therapies deep inside the human body. He holds funding for the development of instrumentation that delivers stem cells to diseased retinal layers in the eye. He and his team are very active in public engagement and patient involvement activities, such as New Scientist Live and the Royal Society Summer Science Exhibition.

View Details

Claire chatted to Mini Rai from Orbit Rise about orbital and planetary robots.

Mini Rai is the founding Director of Orbit Rise Ltd and an honorary Professor at the University of Lincoln. Previously, she was the Global Chair in Robotic Engineering at the University of Lincoln and an Associate Professor at the Surrey Space Centre. Mini has over 27 years of research and innovation experience in Space Engineering and Technology. With deep-rooted expertise in robotics, automation, control and systems engineering, she has led a large and diverse portfolio of national and international programmes on space robotic missions, spanning orbital and planetary robotics.

View Details

Claire chatted to Joe Wolfel from Terradepth about autonomous submersible robots for collecting ocean data.

Joe Wolfel is the CEO and founder of Terradepth. He is passionate about helping people make better and faster decisions regarding what we do (and don’t do) in the ocean. Terradepth designs and builds ocean-going robots at scale, deploys them, and delivers data through an ocean data platform tailored for the maritime community. Prior to Terradepth, Joe has helped start a couple other companies, worked as a management consultant, and served as a US Navy SEAL officer with deployments throughout the Middle East and Africa. Joe was educated at the US Naval Academy.

View Details

Claire chatted to Gabriella Pizzuto from the University of Liverpool about intelligent robotic manipulators for laboratory automation.

Gabriella Pizzuto is a Lecturer in Robotics and Chemistry Automation at the University of Liverpool. She is also a Royal Academy of Engineering Research Fellow and ECR Co-Chair on the EPSRC AI Hub in Chemistry. She has a Ph.D. in Computer Science from the University of Manchester, where she was also a Marie-Sklodowska Curie early stage researcher and a visiting scholar at the University of Edinburgh and Italian Institute of Technology. She was then a postdoctoral research associate at the Edinburgh Centre for Robotics, prior to joining the University of Liverpool.

View Details

On Friday 22 November, IEEE Robotics and Automation Society will be hosting an online science communication training session for robotics and AI researchers. The tutorial will introduce you to science communication and help you create your own story through hands-on activities.

Date: 22 November 2024
Time: 10:00 – 13:00 EST (07:00 – 10:00 PST, 15:00 – 18:00 GMT, 16:00 – 19:00 CET)
Location: Online – worldwide
Registration
Website

Science communication is essential. It helps demystify robotics and AI for a broad range of people including policy makers, business leaders, and the public. As a researcher, mastering this skill can not only enhance your communication abilities but also expand your network and increase the visibility and impact of your work.

In this three-hour session, leading science communicators in robotics and AI will teach you how to clearly and concisely explain your research to non-specialists. You’ll learn how to avoid hype, how to find suitable images and videos to illustrate your work, and where to start with social media. We’ll hear from a leading robotics journalist on how to deal with media and how to get your story out to a wider audience.

This is a hands-on session with exercises for you to take part in throughout the course. Therefore, please come prepared with an idea about a piece of research you’d like to communicate about.

AgendaPart 1: How to communicate your work to a broader audience

• The importance of science communication
• How to produce a short summary of your research for communication via social media channels
• How to expand your outline to write a complete blog post
• How to find and use suitable images
• How to avoid hype when communicating your research
• Unconventional ways of doing science communication

Part 2: How to make videos about your robots

• The value of video
• Tips on making a video

Part 3: Working with media

• Why bother talking to media anyway?
• How media works and what it’s good and bad at
• How to pitch media a story
• How to work with your press office

Speakers:
Sabine Hauert, Professor of Swarm Engineering, Executive Trustee AIhub / Robohub
Lucy Smith, Senior Managing Editor AIhub / Robohub
Laura Bridgeman, Audience Development Manager IEEE Spectrum
Evan Ackerman, Senior Editor IEEE Spectrum

Sign up here.

View Details

Claire chatted to Pratap Tokekar from the University of Maryland about how teams of robots with different capabilities can work together.

Pratap Tokekar is an Associate Professor in the Department of Computer Science and the Institute for Advanced Computer Studies at the University of Maryland, and an Amazon Scholar. Previously, he was a Postdoctoral Researcher at the GRASP lab of University of Pennsylvania and later, an Assistant Professor at Virginia Tech. He has a degree in Electronics and Telecommunication from the College of Engineering Pune in India and a Ph.D. in Computer Science from the University of Minnesota. He received the Amazon Research Award in 2022, and the NSF CAREER award in 2020.

View Details

Claire chatted to Maria Elena Giannaccini from the University of Aberdeen about soft and bioinspired robotics for healthcare and beyond.

Maria Elena Giannaccini has a degree in Biomedical Engineering from the Università di Pisa in Italy. She conducted her Master’s thesis at Scuola Superiore Sant’Anna as part of the EU-funded OCTOPUS project. In 2015, she obtained her PhD in Robotics at the Bristol Robotics, where she focussed on developing safe, variable stiffness robotic devices. She worked at the University of Bristol on the soft, bioinspired Tactip sensor and a soft robotics artificial larynx. In 2019, Elena was appointed as a Lecturer in Robotics at the University of Aberdeen where she pioneered research in soft robotics.

View Details

Claire chatted to Jonathan Walker from Innovate UK about translating robotics research into the commercial sector.

Jonathan Walker is the Innovation Lead for Robotics and Sensors at Innovate UK. He is working with government, universities, businesses and cross-sector teams to accelerate the development and uptake of robotics in the UK. Areas of particular interest are the built environment, circular economy, and helping people live independently for longer. Jonathan wants to support these themes through cluster development, funded collaborative R&D, skills programs from school outreach, apprenticeships and T-levels to CDTs, business support and leveraging private investment.

View Details

Claire chatted to Esyin Chew from Cardiff Metropolitan University about service and social humanoid robots in healthcare and education.

Esyin Chew is the Director of the EUREKA Robotics Centre, one of 11 specialist robotics centres in the UK, impacting underprivileged communities with over 120 humanoid robots. She has led million-pound government or industrial-funded global projects across the UK, EU, Australia, Malaysia, China and Indonesia, including the British Council award-winning Global PIE programme for Women in STEAM-H. Esyin has impacted numerous underprivileged communities, particularly girls and women in education and healthcare sectors, refugees and Orang Asli (Indigenous people).

View Details

Claire chatted to Matt Beane from the University of California, Santa Barbara about how humans can learn to work with intelligent machines.

Matt Beane conducts field research on robots and AI in the workplace, focusing on positive exceptions applicable to the broader world of work. He has published his award-winning research in top management journals and presented on the TED stage. He’s been recognized as a Human-Robot Interaction Pioneer and named to the Thinkers50 Radar list. Matt is an assistant professor in the Technology Management department at the University of California, Santa Barbara, and a Digital Fellow with Stanford’s Digital Economy Lab and MIT’s Initiative on the Digital Economy.

View Details

Claire chatted to Gisela Reyes-Cruz from the University of Nottingham about how humans interact with, trust and accept robots.

Gisela Reyes-Cruz investigates human-computer and human-robot interaction to understand everyday life interactions with technologies, as well as trust in them and public acceptance. These technologies include autonomous and robotic systems: from mobile apps that have, or may have, a component that works autonomously, to robots that can navigate a physical space on their own, such as telepresence robots. The goal of Gisela’s work is to inform responsible system design and design practices.

View Details

Claire chatted to John Leonard from Massachusetts Institute of Technology about autonomous navigation for underwater vehicles and self-driving cars.

John Leonard is a Professor of Mechanical and Ocean Engineering at Massachusetts Institute of Technology (MIT) and a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). His research addresses the problems of navigation and mapping for autonomous underwater vehicles, self-driving vehicles, and other types of mobile robots. He has a degree in Electrical Engineering and Science from the University of Pennsylvania and PhD in Engineering Science from the University of Oxford. He is a Technical Advisor at Toyota Research Institute.

View Details

At the International Joint Conference on Artificial Intelligence (IJCAI) 2023, I had the opportunity to interview Jerry Tan from Lattel Robotics, a company dedicated to promoting AI-focused robotics education and training. They work closely with the RoboCup@Home Education initiative, supporting schools and institutions in introducing AI and service robot development to students. Their goal is to equip learners with practical AI application skills in computer vision, autonomous navigation, object manipulation and speech interactions.

Through their AI robotics and AI applications workshops, Lattel Robotics offers an introduction to robot operating system (ROS)-based AI applications development in service robotics. As a hardware partner for the RoboCup@Home Education initiative, they assist schools and institutions in competing in AI robotic challenges by developing applications that address real-world problems. Their AI learning solutions include setting up AI laboratories, designing curriculums and developing courseware.

Andrea: Thank you for taking the time to speak with me. It’s a pleasure to meet you. Could you please tell me more about who can benefit from the Jupiter2 platform, and what exactly it is designed to do?

Jerry: Just about anyone can use it with a programming background to get started to develop their own AI applications via service robot development, using open source software.

Andrea: What kind of sensors do your robots use? Just microphones and cameras, or is there more?

Jerry: Yes, so we use a range of sensors, including RGB-D cameras and LIDAR sensors for depth perception. Besides, we have microphones and speakers for speech interaction, along with robotic arms for object manipulation and mobile platforms for navigation.

Andrea: So, I have a little question about this. Do you have an API (Application Programming Interface) that students and researchers can work through?

Jerry: It’s not always necessary, as the robot is equipped with its own laptop, acting as a central processing unit. Getting started is easier, as we have integrated the hardware platform with open-source software like OpenCV and YOLO, using the Robot Operating System (ROS1 and ROS2). These are popular tools among AI researchers and developers.

Andrea: It sounds intriguing, but the setup process seems complicated. Integration often takes time.

Jerry: The robotic platform is already integrated and ready to be deployed. For our AI learning device, Juno2, all you need to do is to connect it to a laptop via a USB port and you can straight away boot up the Linux (Ubuntu) environment with ROS, without any software installation.

AI learning device, Juno2.

Andrea: How do you work with schools?

Jerry: We’re an education solution provider, supporting schools and institutions that want to introduce practical AI applications learning in a classroom. Programming experience in Python or C++ is a prerequisite in doing AI applications development. So we provide Python introduction workshop for complete beginners. Then using a train-the-trainers approach, we offer 3-day AI robotics workshops using Jupiter2 or 2-day AI applications workshops using Juno2. Through these workshops, educators are given an overview of the AI applications modules which would enable them to customise the material accordingly for education, training or even research purposes.

Andrea: What if someone doesn’t have direct access to the robot? Can they work remotely?

Jerry: Absolutely. You can connect via a remote desktop software from your computer and continue programming. As mentioned just now, we also have a smaller device called Juno2, designed for classroom use. You don’t need the full robot setup in this case; you can use this device to do computer vision and speech interactions applications. It’s a plug-and-play solution that works with any laptop or operating system, automatically loading Ubuntu, ROS and our Jupiter software development frameworks. It’s perfect for conducting online classes, as students can continue learning from home or wherever they are.

Andrea: If you have a robot like this, can additional components, like speakers, be added to your robots?

Jerry: Yes, definitely. That’s a good question. Jupiter2 is an open platform that is
customisable and reconfigurable. If you have an engineering background, you can modify the hardware based on the requirements. This flexibility allows both beginners and advanced users to focus on their specific areas of interest, whether it is software programming or hardware customisation.

Andrea: Have you exported these gadgets before?

Jerry: Yes, we have exported to Europe, Africa, South and North America before. So far there are more than 30 over schools, universities and institutions globally that have adopted our learning platforms for education, training, research and/or robotic competitions.

Andrea: Fantastic, thank you for your time and insights. I wish you continued success in the future!

You can find more information on the Lattel Robotics website.

| | Jerry Tan is the Managing Director of Lattel Robotics. A German-trained engineer turned entrepreneur, Jerry is currently running the AI robotics education and training companies in Malaysia and Singapore. Striving to empower anyone in getting started to learn AI applications practically, Jerry works closely with schools and academic institutions in setting up laboratories and developing courseware. |

View Details

A break in play during a Small Size League match.

Today, 21 July, saw the competitions draw to a close in a thrilling finale. In the third and final of our round-up articles, we provide a flavour of the action from this last day. If you missed them, you can find our first two digests here: 19 July | 20 July.

My first port of call this morning was the Standard Platform League, where Dr Timothy Wiley and Tom Ellis from Team RedbackBots, RMIT University, Melbourne, Australia, demonstrated an exciting advancement that is unique to their team. They have developed an augmented reality (AR) system with the aim of enhancing the understanding and explainability of the on-field action.

The RedbackBots travelling team for 2024 (L-to-R: Murray Owens, Sam Griffiths, Tom Ellis, Dr Timothy Wiley, Mark Field, Jasper Avice Demay). Photo credit: Dr Timothy Wiley.

Timothy, the academic leader of the team explained: “What our students proposed at the end of last year’s competition, to make a contribution to the league, was to develop an augmented reality (AR) visualization of what the league calls the team communication monitor. This is a piece of software that gets displayed on the TV screens to the audience and the referee, and it shows you where the robots think they are, information about the game, and where the ball is. We set out to make an AR system of this because we think it’s so much better to view it overlaid on the field. What the AR lets us do is project all of this information live on the field as the robots are moving.”

The team has been demonstrating the system to the league at the event, with very positive feedback. In fact, one of the teams found an error in their software during a game whilst trying out the AR system. Tom said that they’ve received a lot of ideas and suggestions from the other teams for further developments. This is one of the first (if not, the first) AR system to be trialled across the competition, and first time it has been used in the Standard Platform League. I was lucky enough to get a demo from Tom and it definitely added a new level to the viewing experience. It will be very interesting to see how the system evolves.

Mark Field setting up the MetaQuest3 to use the augmented reality system. Photo credit: Dr Timothy Wiley.

From the main soccer area I headed to the RoboCupJunior zone, where Rui Baptista, an Executive Committee member, gave me a tour of the arenas and introduced me to some of the teams that have been using machine learning models to assist their robots. RoboCupJunior is a competition for school children, and is split into three leagues: Soccer, Rescue and OnStage.

I first caught up with four teams from the Rescue league. Robots identify “victims” within re-created disaster scenarios, varying in complexity from line-following on a flat surface to negotiating paths through obstacles on uneven terrain. There are three different strands to the league: 1) Rescue Line, where robots follow a black line which leads them to a victim, 2) Rescue Maze, where robots need to investigate a maze and identify victims, 3) Rescue Simulation, which is a simulated version of the maze competition.

Team Skollska Knijgia, taking part in the Rescue Line, used a YOLO v8 neural network to detect victims in the evacuation zone. They trained the network themselves with about 5000 images. Also competing in the Rescue Line event were Team Overengeniering2. They also used YOLO v8 neural networks, in this case for two elements of their system. They used the first model to detect victims in the evacuation zone and to detect the walls. Their second model is utilized during line following, and allows the robot to detect when the black line (used for the majority of the task) changes to a silver line, which indicates the entrance of the evacuation zone.

Left: Team Skollska Knijgia. Right: Team Overengeniering2.

Team Tanorobo! were taking part in the maze competition. They also used a machine learning model for victim detection, training on 3000 photos for each type of victim (these are denoted by different letters in the maze). They also took photos of walls and obstacles, to avoid mis-classification. Team New Aje were taking part in the simulation contest. They used a graphical user interface to train their machine learning model, and to debug their navigation algorithms. They have three different algorithms for navigation, with varying computational cost, which they can switch between depending on the place (and complexity) in the maze in which they are located.

Left: Team Tanorobo! Right: Team New Aje.

I met two of the teams who had recently presented in the OnStage event. Team Medic’s performance was based on a medical scenario, with the team including two machine learning elements. The first being voice recognition, for communication with the “patient” robots, and the second being image recognition to classify x-rays. Team Jam Session’s robot reads in American sign language symbols and uses them to play a piano. They used the MediaPipe detection algorithm to find different points on the hand, and random forest classifiers to determine which symbol was being displayed.

Left: Team Medic Bot Right: Team Jam Session.

Next stop was the humanoid league where the final match was in progress. The arena was packed to the rafters with crowds eager to see the action.
Standing room only to see the Adult Size Humanoids.

The finals continued with the Middle Size League, with the home team Tech United Eindhoven beating BigHeroX by a convincing 6-1 scoreline. You can watch the livestream of the final day’s action here.

The grand finale featured the winners of the Middle Size League (Tech United Eindhoven) against five RoboCup trustees. The humans ran out 5-2 winners, their superior passing and movement too much for Tech United.

View Details

The Standard Platform Soccer League in action.

This is the second of our daily digests from RoboCup2024 in Eindhoven, The Netherlands. If you missed the first digest, which gives some background to RoboCup, you can find it here.

Competitions continued across all the leagues today, with participants vying for a place in Sunday’s finals.

The RoboCup@Work league focusses on robots in work-related scenarios, utilizing ideas and concepts from other RoboCup competitions to tackle open research challenges in industrial and service robotics.

I arrived at the arena in time to catch the advanced navigation test. Robots have to autonomously navigate, picking up and placing objects at different work stations. In this advanced test, caution tape is added to the arena floor, which the robots should avoid travelling over. There is also a complex placing element where teams have to put an object that they’ve collected into a slot – get the orientation or placement of the object slightly wrong and the it won’t fall into the slot.

The RoboCup@Work arena just before competition start.

Eight teams are taking part in the league this year. Executive Committee member Asad Norouzi said that there are plans to introduce a sub-league which would provide an entry point for new teams or juniors to get into the league proper.

I caught up with Harrison Burns, Mitchell Torok and Jasper Arnold from Team MiRobot. They are based at the University of New South Wales and are attending RoboCup for the first time.

Team MiRobot from UNSW.

The team actually only started six months ago, so final preparations have been a bit stressful. However, the experience has been great fun, and the competition has gone well so far. Like most teams, they’ve had to make many refinements as the competition has progressed, leading to some late nights.

One notable feature of the team’s robot is the bespoke, in-house-designed grasping mechanism on the end of the arm. The team note that “it has good flexible jaws, so when it grabs round objects it actually pulls the object directly into it. Because it uses a linear motion, compared to a lot of other rotating jaws, it has a lot better reliability for picking up objects”.

Here is some footage from the task, featuring Team bi-t-bots and Team Singapore.

Team b-it-bots take on the RoboCup@Work advanced navigation test, picking up a drill bit #RoboCup2024 pic.twitter.com/QfijZpxaOK

— AIhub (@aihuborg) July 20, 2024

Team Singapore placing an object in the RoboCup@Work advanced navigation test pic.twitter.com/SvgLBOaVo7

— AIhub (@aihuborg) July 20, 2024

In the Middle Size Soccer league (MSL), teams of five fully autonomous robots play with a regular size FIFA ball. Teams are free to design their own hardware but all sensors have to be on-board and there is a maximum size and weight limit of 40kg for the robots. The research focus is on mechatronics design, control and multi-agent cooperation at plan and perception levels. Nine teams are competing this year.

Action from the Middle Size League at #RoboCup2024.

Falcons vs Robot Club Toulon pic.twitter.com/GHcNLOx2nV

— AIhub (@aihuborg) July 20, 2024

I spoke to António Ribeiro, who is a member of the technical committee and part of Team LAR@MSL from the University of Minho, Portugal. The team started in 1998, but António and most of his colleagues on the current team have only been involved in the MSL since September 2022. The robots have evolved as the competition has progressed, and further improvements are in progress. Refinements so far have included communication, the detection system, and the control system. They are pleased with the improvements from the previous RoboCup. “Last year we had a lot of hardware issues, but this year the hardware seems pretty stable. We also changed our coding architecture and it is now much easier and faster for us to develop code because we can all work on the code at the same time on different modules”.

António cited versatility and cost-effective solutions as strengths of the team. “Our robot is actually very cheap compared to other teams. We use a lot of old chassis, and our solutions always go to the lowest cost possible. Some teams have multiple thousand dollar robots, but, for example, our vision system is around $70-80. It works pretty well – we need to improve the way we handle it, but it seems stable”.

Team LAR@MSL

The RoboCup@Home league aims to develop service and assistive robot technology with high relevance for future personal domestic applications. A set of benchmark tests is used to evaluate the robots’ abilities and performance in a realistic non-standardized home environment setting. These tests include helping to prepare breakfast, clearing the table, and storing groceries.

I arrived in time to watch the “stickler for the rules” challenge, where robots have to navigate different rooms and make sure that the people inside (“guests” at a party) are sticking to four rules: 1) there is one forbidden room – if a guest is in there the robot must alert them and ask them to follow it into another room), 2) everyone must have a drink in their hand – if not, the robot directs them to a shelf with drinks, 3) no shoes to be worn in the house, 4) there should be no rubbish left on the floor.

After watching an attempt from the LAR@Home robot, Tiago from the team told me a bit about the robot. “The goal is to develop a robot capable of multi general-purpose tasks in home and healthcare environments.” With the exception of the robotic arm, all of the hardware was built by the team. The robot has two RGBD cameras, two LIDARs, a tray (where the robot can store items that it needs to carry), and two emergency stop buttons that deactivate all moving parts. Four omnidirectional wheels allow the robot to move in any direction at any time. The wheels have independent suspension systems which guarantees that they can all be on the ground at all times, even if there are bumps and cables on the venue floor. There is a tablet that acts as a visual interface, and a microphone and speakers to enable communication between humans and the robot, which is all done via speaking and listening.

Tiago told me that the team have talked to a lot healthcare practitioners to find out the main problems faced by elderly people, and this inspired one of their robot features. “They said that the two main injury sources are from when people are trying to sit down or stand up, and when they are trying to pick something up from the floor. We developed a torso that can pick objects from the floor one metre away from the robot”.

The LAR@Home team.

You can keep up with the latest news direct from RoboCup here.

Click here to see all of our content pertaining to RoboCup.

View Details

The main soccer arena.

RoboCup is an international scientific initiative with the goal to advance the state of the art of intelligent robots. As part of this initiative, a series of competitions and events are held throughout the year. The main showcase event is an international affair with teams travelling from far and wide to put their machines through their paces.

This year, RoboCup is being held in three arenas in the Genneper Parken, Eindhoven, The Netherlands. The organisers are expecting over 2,000 participants, from 45 different countries, with around 300 teams signed up to take part in the various competitions.

Although RoboCup started out as a football (or soccer) playing competition, other leagues have since been introduced, focussing on robots in industrial, rescue, and home settings. There is even a dedicated league for young roboticists – RoboCupJunior – where participants can take part in either football, rescue, or artistic events.

I am lucky enough to be able to attend this year, and, for the next three days, I’ll be bringing you a daily digest of some of the exciting happenings from Eindhoven.

Today, 19 July, sees the competition in full swing. The main soccer arena, boasting multiple pitches, hosts a number of the different leagues which form RoboCupSoccer.

Some of the pitches in the main soccer arena.

My first port of call was the Standard Platform League, where the round 5 champions cup match between SPQR Team vs rUNSWift was taking place. SPQR ran out winners and advance to round 6. In this league, all teams compete with identical robots (currently the humanoid NAO by Aldebaran). The robots operate fully autonomously, meaning that there is no external control from neither humans nor computers.

Standard platform league. Round 5 champions cup match between SPQR Team vs rUNSWift.

Goal! pic.twitter.com/dMfNDUKNZc

— AIhub (@aihuborg) July 19, 2024

The Humanoid AdultSize league is arguably the most challenging of the leagues, with many constraints placed on the robots to make them as human-like as possible. For example, they must have roughly human-like body proportions, they need to walk on two legs, and they are only allowed to use human-like sensors (up to two cameras to sense the environment). In this AdultSize competition, two robots from each team compete, and the team members walk behind the robots to catch them in case of a fall. Such a mishap could prove costly in terms of potential hardware damage.

Action from the Humanoid AdultSize League.

The RoboCup Rescue Robot League sees teams developing robotic systems with the goal of enabling emergency responders to perform extremely hazardous tasks from safer stand-off distances. During the competition, teams compete in a round-robin, putting their robots through their paces on a number of different challenges. The leading teams following this initial phase progress to the finals on Sunday. The tasks include navigating in complex environments, opening doors, and sensing. Teams may run the machines completely autonomously, or with some assistive control. More points are awarded for completely autonomous operation.

RoboCup Rescue arena from above.

Some action from the @robocup_org #RoboCup2024 Rescue league, where teams compete in a variety of challenges.

Team Hector Darmstadt in the "Obstacles: pallets with pipes" challenge pic.twitter.com/4Ll75uENjM

— AIhub (@aihuborg) July 19, 2024

KMUTNB navigate rough terrain, including gravel and sand pic.twitter.com/rsI7NliEwd

— AIhub (@aihuborg) July 19, 2024

You can keep up with more RoboCup2024 news here.

View Details

Robotics is helping to rehabilitate and increase human abilities in areas like mobility and stamina. Innovations in robotic devices, exoskeletons, and wearable tech aim to offer disabled people different perspectives and new experiences, as well as supporting humans more widely to access, inhabit and work safely in dangerous and extreme conditions. What does the future hold for these technologies and the people they will become a part of?

In this special live recording at the Victoria and Albert Museum as part of the Great Exhibition Road Festival, Claire chatted to Milia Helena Hasbani (Imperial College London), Benjamin Metcalfe (University of Bath) and Dani Clode (Cambridge University) about robotic prosthetics and human augmentation.

Milia Helena Hasbani is a researcher in assistive technology at Imperial College London. She is passionate about improving people’s lives through innovation in healthcare and technology in multi-disciplinary environments interfacing with engineers, clinicians, and patients. Her research focuses on the control of active prosthetic arms, combining user intention for wrist movements with a computer vision system for dynamically selecting the grasp type to be used. Benjamin Metcalfe is a biomedical engineer who specialises in neural interfaces and implanted devices. He is Head of the Department of Electronic & Electrical Engineering at the University of Bath and Deputy Director of the Bath Institute for the Augmented Human. He is also Vice-President (Academic) of the Institute of Physics and Engineering in Medicine. His interests explore the collision between technology and biology and the extent to which engineering can be used to augment and enhance human performance. Dani Clode is an augmentation and prosthetics designer. She is the Senior Technical Specialist at the Plasticity Lab at Cambridge University and a collaborator of the Alternative Limb Project. Dani’s work investigates the future architecture of our bodies, challenging the perception and boundaries of extending the human form. Her main project the ‘Third Thumb’ is currently being utilised in collaboration with neuroscientists at Cambridge University, investigating the brain’s ability to adapt to human augmentation.

View Details

Claire chatted to Simone Schuerle from ETH Zürich all about microrobots, medicine and science.

Simone Schuerle is Assistant Professor at ETH Zurich in Switzerland, where she heads the Responsive Biomedical System Lab. With her team, she develops diagnostic and therapeutic systems at the nano- and microscale with the aim of tackling a range of challenging problems in medicine. One major focus of her current research is addressing limitations in drug delivery through scalable magnetically enhanced drug transport. In 2014, she co-founded the spin-off MagnebotiX that offers electromagnetic control systems for wireless micromanipulation.

View Details

Claire chatted to Lord Ara Darzi from Imperial College London all about robotic surgery – past, present and future.

Ara Darzi is Co-Director of the Institute of Global Health Innovation at Imperial College London and holds the Paul Hamlyn Chair of Surgery. In 2002, he was knighted for his services to medicine and surgery and in 2007 was introduced as Lord Darzi of Denham to the UK’s House of Lords as the Parliamentary Under-Secretary of State for Health. Professor Darzi leads a large multidisciplinary academic and policy research team, focused on convergence science across engineering, physical and data sciences, specifically in the areas of robotics, sensing, imaging and digital and AI technologies. He is Chair of the NHS Accelerated Access Collaborative, Fellow of the Academy of Medical Sciences and the Royal Society, and Honorary Fellow of the Royal Academy of Engineering.

View Details

Claire chatted to Isabelle Ormerod from the University of Bristol all about human-centred design and women in robotics.

Isabelle Ormerod is a PhD student at Bristol Robotics Lab in the Design and Manufacturing Futures Lab. Her professional path began in the medical product design industry, where she observed firsthand the application of human factors (HF) processes for dextrous and high-risk procedures. This experience sparked her interest in leveraging data-driven HF approaches in product design. Isabelle is also part of the Leadership team of Women in Robotics UK. This organization is committed to fostering an inclusive community for women and non-binary individuals in the robotics field in the UK.

View Details

Claire chatted to Mario Di Castro from CERN all about robotic inspection and maintenance in hazardous environments.

Mario Di Castro has a Master’s degree in electronic engineering from the University of Naples Federico II in Italy and a PhD in robotics and industrial controls from the Polytechnic University of Madrid in Spain. Since 2011 he has led the Mechatronics, Robotics and Operation section at CERN. The section is responsible for the design, construction, installation, operation and maintenance of robotic systems used for remote maintenance at the CERN accelerator complex. His research interests include tele-robotics, machine learning, and precise motion control in harsh environments.

View Details

The 2024 IEEE International Conference on Robotics and Automation (ICRA) best paper winners and finalists in the various different categories have been announced. The recipients were revealed during an award luncheon at the conference, which took place from 13-17 May in Yokohama, Japan.

IEEE ICRA Best Paper Award in AutomationWinnerTinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers, Anoushka Alavilli, Khai Nguyen, Samuel Schoedel, Brian Plancher, and Zachary Manchester

Finalists A Movable Microfluidic Chip with Gap Effect for Manipulation of Oocytes, Shuzhang Liang, Satoshi Amaya, Hirotaka Sugiura, Hao Mo, Yuguo Dai, and Fumihito Arai * Under Pressure: Learning-Based Analog Gauge Reading in the Wild, Maurits Reitsma, Julian Keller, Kenneth Blomqvist, and Roland Siegwart * Efficient Composite Learning Robot Control Under Partial Interval Excitation, Tian Shi, Weibing Li, Haoyong Yu, and Yongping Pan * MORALS: Analysis of High-Dimensional Robot Controllers via Topological Tools in a Latent Space, Ewerton Vieira, Aravind Sivaramakrishnan, Sumanth Tangirala, Edgar Granados, Konstantin Mischaikow, and Kostas E. Bekris*

IEEE ICRA Best Paper Award in Cognitive RoboticsWinnerVLFM: Vision-Language Frontier Maps for Semantic Navigation, Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher

Finalists NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration, Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine * Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End, Chong Zhang, Jin Jin, Jonas Frey, Nikita Rudin, Matias Mattamala, Cesar Cadena Lerma, and Marco Hutter * Learning Continuous Control with Geometric Regularity from Robot Intrinsic Symmetry, Shengchao Yan, Baohe Zhang, Yuan Zhang, Joschka Boedecker, and Wolfram Burgard * Learning Vision-Based Bipedal Locomotion for Challenging Terrain, Helei Duan, Bikram Pandit, Mohitvishnu S. Gadde, Bart Jaap Van Marum, Jeremy Dao, Chanho Kim, and Alan Fern*

IEEE ICRA Best Paper Award in Robot ManipulationWinnerSARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention, Isabel Leal, Krzysztof Choromanski, Deepali Jain, Avinava Dubey, Jacob Varley, Michael S. Ryoo, Yao Lu, Frederick Liu, Vikas Sindhwani, Tamas Sarlos, Kenneth Oslund, Karol Hausman, Quan Vuong, and Kanishka Rao

Finalists Open X-Embodiment: Robotic Learning Datasets and RT-X Models, Sergey Levine, Chelsea Finn, Ken Goldberg, Lawrence Yunliang Chen, Gaurav Sukhatme, Shivin Dass, Lerrel Pinto, Yuke Zhu, Yifeng Zhu, Shuran Song, Oier Mees, Deepak Pathak, Hao-Shu Fang, Henrik Iskov Christensen, Mingyu Ding, Youngwoon Lee, Dorsa Sadigh, Ilija Radosavovic, Jeannette Bohg, Xiaolong Wang, Xuanlin Li, Krishan Rana, Kento Kawaharazuka, Tatsuya Matsushima, Jihoon Oh, Takayuki Osa, Oliver Kroemer, Beomjoon Kim, Edward Johns, Freek Stulp, Jan Schneider, Jiajun Wu, Yunzhu Li, Heni Ben Amor, Lionel Ott, Roberto Martin-Marin, Karol Hausman, Quan Vuong, Pannag Sanketi, Nicolas Heess, Vincent Vanhoucke, Karl Pertsch, Stefan Schaal, Cheng Chi, Chuer Pan, and Alex Bewley * Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans, Homanga Bharadhwaj, Abhinav Gupta, Vikash Kumar, and Shubham Tulsiani * Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation, Jared Mejia, Victoria Dean, Tess Hellebrekers, and Abhinav Gupta * DenseTact-Mini: An Optical Tactile Sensor for Grasping Multi-Scale Objects From Flat Surfaces, Won Kyung Do, Ankush Ankush Dhawan, Mathilda Kitzmann, and Monroe Kennedy * Constrained Bimanual Planning with Analytic Inverse Kinematics, Thomas Cohn, Seiji Shaw, Max Simchowitz, and Russ Tedrake*

IEEE ICRA Best Paper Award on Human-Robot InteractionWinnerCoFRIDA: Self-Supervised Fine-Tuning for Human-Robot Co-Painting, Peter Schaldenbrand, Gaurav Parmar, Jun-Yan Zhu, James Mccann, and Jean Oh

Finalists POLITE: Preferences Combined with Highlights in Reinforcement Learning, Simon Holk, Daniel Marta, and Iolanda Leite * MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments, Junwei Zheng, Jiaming Zhang, Kailun Yang, Kunyu Peng, and Rainer Stiefelhagen * Robot-Assisted Navigation for Visually Impaired through Adaptive Impedance and Path Planning, Pietro Balatti, Idil Ozdamar, Doganay Sirintuna, Luca Fortini, Mattia Leonori, Juan M. Gandarias, and Arash Ajoudani * Incremental Learning of Full-Pose Via-Point Movement Primitives on Riemannian Manifolds, Tilman Daab, Noémie Jaquier, Christian R. G. Dreher, Andre Meixner, Franziska Krebs, and Tamim Asfour * Supernumerary Robotic Limbs to Support Post-Fall Recoveries for Astronauts, Erik Ballesteros, Sang-Yoep Lee, Kalind Carpenter, and Harry Asada*

IEEE ICRA Best Paper Award in Medical RoboticsWinnerExoskeleton-Mediated Physical Human-Human Interaction for a Sit-to-Stand Rehabilitation Task, Lorenzo Vianello, Emek Baris Kucuktabak, Matthew Short, Clément Lhoste, Lorenzo Amato, Kevin Lynch, and Jose L. Pons

Finalists Intraoperatively Iterative Hough Transform Based In-plane Hybrid Control of Arterial Robotic Ultrasound for Magnetic Catheterization, Zhengyang Li, Magejiang Yeerbulati, and Qingsong Xu * Efficient Model Learning and Adaptive Tracking Control of Magnetic Micro-Robots for Non-Contact Manipulation, Yongyi Jia, Shu Miao, Junjian Zhou, Niandong Jiao, Lianqing Liu, and Xiang Li * Colibri5: Real-Time Monocular 5-DoF Trocar Pose Tracking for Robot-Assisted Vitreoretinal Surgery, Shervin Dehghani, Michael Sommersperger, Mahdi Saleh, Alireza Alikhani, Benjamin Busam, Peter Gehlbach, Ioan Iulian Iordachita, Nassir Navab, and M. Ali Nasseri * Hybrid Volitional Control of a Robotic Transtibial Prosthesis using a Phase Variable Impedance Controller, Ryan Posh, Jonathan Allen Tittle, David Kelly, James Schmiedeler, and Patrick M. Wensing * Design and Implementation of a Robotized Hand-held Dissector for Endoscopic Pulmonary Endarterectomy, Runfeng Zhu, Xilong Hou, Wei Huang, Lei Du, Zhong Wu, Hongbin Liu, Henry Chu, and Qing Xiang Zhao*

IEEE ICRA Best Paper Award on Mechanisms and DesignWinnerDesign and Modeling of a Nested Bi-cavity-based Soft Growing Robot for Grasping in Constrained Environments, Haochen Yong, Fukang Xu, Chenfei Li, Han Ding, and Zhigang Wu

Finalists Optimized Design and Fabrication of Skeletal Muscle Actuators for Bio-syncretic Robots, Lianchao Yang, Chuang Zhang, Ruiqian Wang, Yiwei Zhang, and Lianqing Liu * Lissajous Curve-Based Vibrational Orbit Control of a Flexible Vibrational Actuator with a Structural Anisotropy, Yuto Miyazaki and Mitsuru Higashimori * Dynamic Modeling of Wing-Assisted Inclined Running with a Morphing Multi-Modal Robot, Eric Sihite, Alireza Ramezani, and Gharib Morteza*

IEEE ICRA Best Paper Award on Multi-Robot SystemsWinnerDo We Run Large-Scale Multi-Robot Systems on the Edge? More Evidence for Two-Phase Performance in System Size Scaling, Jonas Kuckling, Robin Luckey, Viktor Avrutin, Andrew Vardy, Andreagiovanni Reina, and Heiko Hamann

Finalists Observer-based Distributed MPC for Collaborative Quadrotor-Quadruped Manipulation of a Cable-Towed Load, Shaohang Xu, Yi’An Wang, Wentao Zhang, Chin Pang Ho, and Lijun Zhu * Learning for Dynamic Subteaming and Voluntary Waiting in Heterogeneous Multi-Robot Collaborative Scheduling, Williard Joshua Jose and Hao Zhang * Asynchronous Distributed Smoothing and Mapping via On-Manifold Consensus ADMM, Daniel Mcgann, Kyle Lassak, and Michael Kaess * Uncertainty-Bounded Active Monitoring of Unknown Dynamic Targets in Road-Networks with Minimum Fleet, Shuaikang Wang, Yiannis Kantaros, and Meng Guo*

IEEE ICRA Best Paper Award in Service RoboticsWinnerLearning to Walk in Confined Spaces Using 3D Representation, Takahiro Miki, Joonho Lee, Lorenz Wellhausen, and Marco Hutter

Finalists Censible: A Robust and Practical Global Localization Framework for Planetary Surface Missions, Jeremy Nash, Quintin Dwight, Lucas Saldyt, Haoda Wang, Steven Myint, Adnan Ansar, and Vandi Verma * Efficient and Accurate Transformer-Based 3D Shape Completion and Reconstruction of Fruits for Agricultural Robots, Federico Magistri, Rodrigo Marcuzzi, Elias Ariel Marks, Matteo Sodano, Jens Behley, and Cyrill Stachniss * CoPAL: Corrective Planning of Robot Actions with Large Language Models, Frank Joublin, Antonello Ceravola, Pavel Smirnov, Felix Ocker, Joerg Deigmoeller, Anna Belardinelli, Chao Wang, Stephan Hasler, Daniel Tanneberg, and Michael Gienger * CalliRewrite: Recovering Handwriting Behaviors from Calligraphy Images without Supervision, Yuxuan Luo, Zekun Wu, and Zhouhui Lian*

IEEE ICRA Best Paper Award in Robot VisionWinnerNGEL-SLAM: Neural Implicit Representation-based Global Consistent Low-Latency SLAM System, Yunxuan Mao, Xuan Yu, Kai Wang, Yue Wang, Rong Xiong, and Yiyi Liao

Finalists HEGN: Hierarchical Equivariant Graph Neural Network for 9DoF Point Cloud Registration, Adam Misik, Driton Salihu, Xin Su, Heike Brock, and Eckehard Steinbach * Deep Evidential Uncertainty Estimation for Semantic Segmentation under Out-Of-Distribution Obstacles, Siddharth Ancha, Philip Osteen, and Nicholas Roy * SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking, Yu Lin, Zhiheng Li, Yubo Cui, and Zheng Fang * Ultrafast Square-Root Filter-based VINS, Yuxiang Peng, Chuchu Chen, and Guoquan Huang * Universal Visual Decomposer: Long-Horizon Manipulation Made Easy, Zichen Zhang, Yunshuang Li, Osbert Bastani, Abhishek Gupta, Dinesh Jayaraman, Yecheng Jason Ma, and Luca Weihs*

IEEE ICRA Best Paper Award on Unmanned Aerial VehiclesWinnerTime-Optimal Gate-Traversing Planner for Autonomous Drone Racing, Chao Qin, Maxime Simon Joseph Michet, Jingxiang Chen, and Hugh H.-T. Liu

Finalists A Trajectory-based Flight Assistive System for Novice Pilots in Drone Racing Scenario, Yuhang Zhong, Guangyu Zhao, Qianhao Wang, Guangtong Xu, Chao Xu, and Fei Gao * Co-Design Optimisation of Morphing Topology and Control of Winged Drones, Fabio Bergonti, Gabriele Nava, Valentin Wüest, Antonello Paolino, Giuseppe L’Erario, Daniele Pucci, and Dario Floreano * FC-Planner: A Skeleton-guided Planning Framework for Fast Aerial Coverage of Complex 3D Scenes, Chen Feng, Haojia Li, Mingjie Zhang, Xinyi Chen, Boyu Zhou, and Shaojie Shen * Sequential Trajectory Optimization for Externally-Actuated Modular Manipulators with Joint Locking, Jaeu Choe, Jeongseob Lee, Hyunsoo Yang, Hai-Nguyen (Hann) Nguyen, and Dongjun Lee * Spatial Assisted Human-Drone Collaborative Navigation and Interaction through Immersive Mixed Reality, Luca Morando and Giuseppe Loianno*

IEEE ICRA Best Student Paper AwardWinnerOptimized Design and Fabrication of Skeletal Muscle Actuators for Bio-syncretic Robots, Lianchao Yang, Chuang Zhang, Ruiqian Wang, Yiwei Zhang, and Lianqing Liu

Finalists TinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers, Anoushka Alavilli, Khai Nguyen, Samuel Schoedel, Brian Plancher, and Zachary Manchester * Goal Masked Diffusion Policies for Unified Navigation and Exploration, Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine * Open X-Embodiment: Robotic Learning Datasets and RT-X Models, Sergey Levine, Chelsea Finn, Ken Goldberg, Lawrence Yunliang Chen, Gaurav Sukhatme, Shivin Dass, Lerrel Pinto, Yuke Zhu, Yifeng Zhu, Shuran Song, Oier Mees, Deepak Pathak, Hao-Shu Fang, Henrik Iskov Christensen, Mingyu Ding, Youngwoon Lee, Dorsa Sadigh, Ilija Radosavovic, Jeannette Bohg, Xiaolong Wang, Xuanlin Li, Krishan Rana, Kento Kawaharazuka, Tatsuya Matsushima, Jihoon Oh, Takayuki Osa, Oliver Kroemer, Beomjoon Kim, Edward Johns, Freek Stulp, Jan Schneider, Jiajun Wu, Yunzhu Li, Heni Ben Amor, Lionel Ott, Roberto Martin-Martin, Karol Hausman, Quan Vuong, Pannag Sanketi, Nicolas Heess, Vincent Vanhoucke, Karl Pertsch, Stefan Schaal, Cheng Chi, Chuer Pan, and Alex Bewley * POLITE: Preferences Combined with Highlights in Reinforcement Learning, Simon Holk, Daniel Marta, and Iolanda Leite * Exoskeleton-Mediated Physical Human-Human Interaction for a Sit-to-Stand Rehabilitation Task, Lorenzo Vianello, Emek Baris Kucuktabak, Matthew Short, Clément Lhoste, Lorenzo Amato, Kevin Lynch, and Jose L. Pons * Design and Modeling of a Nested Bi-cavity- based Soft Growing Robot for Grasping in Constrained Environments, Haochen Yong, Fukang Xu, Chenfei Li, Han Ding, and Zhigang Wu * Observer-based Distributed MPC for Collaborative Quadrotor-Quadruped Manipulation of a Cable-Towed Load, Shaohang Xu, Yi’An Wang, Wentao Zhang, Chin Pang Ho, and Lijun Zhu * Censible: A Robust and Practical Global Localization Framework for Planetary Surface Missions, Jeremy Nash, Quintin Dwight, Lucas Saldyt, Haoda Wang, Steven Myint, Adnan Ansar, and Vandi Verma * HEGN: Hierarchical Equivariant Graph Neural Network for 9DoF Point Cloud Registration, Adam Misik, Driton Salihu, Xin Su, Heike Brock, and Eckehard Steinbach * A Trajectory-based Flight Assistive System for Novice Pilots in Drone Racing Scenario, Yuhang Zhong, Guangyu Zhao, Qianhao Wang, Guangtong Xu, Chao Xu, and Fei Gao*

IEEE ICRA Best Conference Paper AwardWinners Goal Masked Diffusion Policies for Unified Navigation and Exploration, Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine * Open X-Embodiment: Robotic Learning Datasets and RT-X, Sergey Levine, Chelsea Finn, Ken Goldberg, Lawrence Yunliang Chen, Gaurav Sukhatme, Shivin Dass, Lerrel Pinto, Yuke Zhu, Yifeng Zhu, Shuran Song, Oier Mees, Deepak Pathak, Hao-Shu Fang, Henrik Iskov Christensen, Mingyu Ding, Youngwoon Lee, Dorsa Sadigh, Ilija Radosavovic, Jeannette Bohg, Xiaolong Wang, Xuanlin Li, Krishan Rana, Kento Kawaharazuka, Tatsuya Matsushima, Jihoon Oh, Takayuki Osa, Oliver Kroemer, Beomjoon Kim, Edward Johns, Freek Stulp, Jan Schneider, Jiajun Wu, Yunzhu Li, Heni Ben Amor, Lionel Ott, Roberto Martin-Martin, Karol Hausman, Quan Vuong, Pannag Sanketi, Nicolas Heess, Vincent Vanhoucke, Karl Pertsch, Stefan Schaal, Cheng Chi, Chuer Pan, and Alex Bewley*

Finalists TinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers, Anoushka Alavilli, Khai Nguyen, Samuel Schoedel, Brian Plancher, and Zachary Manchester * POLITE: Preferences Combined with Highlights in Reinforcement Learning, Simon Holk, Daniel Marta, and Iolanda Leite * Exoskeleton-Mediated Physical Human-Human Interaction for a Sit-to-Stand Rehabilitation Task, Lorenzo Vianello, Emek Baris Kucuktabak, Matthew Short, Clément Lhoste, Lorenzo Amato, Kevin Lynch, and Jose L. Pons * Optimized Design and Fabrication of Skeletal Muscle Actuators for Bio-syncretic Robots, Lianchao Yang, Chuang Zhang, Ruiqian Wang, Yiwei Zhang, and Lianqing Liu * Design and Modeling of a Nested Bi-cavity- based Soft Growing Robot for Grasping in Constrained Environments, Haochen Yong, Fukang Xu, Chenfei Li, Han Ding, and Zhigang Wu * Observer-based Distributed MPC for Collaborative Quadrotor-Quadruped Manipulation of a Cable-Towed Load, Shaohang Xu, Yi’An Wang, Wentao Zhang, Chin Pang Ho, and Lijun Zhu * Censible: A Robust and Practical Global Localization Framework for Planetary Surface Missions, Jeremy Nash, Quintin Dwight, Lucas Saldyt, Haoda Wang, Steven Myint, Adnan Ansar, and Vandi Verma * HEGN: Hierarchical Equivariant Graph Neural Network for 9DoF Point Cloud Registration, Adam Misik, Driton Salihu, Xin Su, Heike Brock, and Eckehard Steinbach * A Trajectory-based Flight Assistive System for Novice Pilots in Drone Racing Scenario, Yuhang Zhong, Guangyu Zhao, Qianhao Wang, Guangtong Xu, Chao Xu, and Fei Gao*

View Details

Claire chatted to Margarita Chli from the University of Cyprus all about vision, navigation, and small aerial drones.

Margarita Chli is a professor of Robotic Vision and the director of the Vision for Robotics Lab, at the University of Cyprus and ETH Zurich. Her work has contributed to the first vision-based autonomous flight of a small drone and the first demonstration of collaborative monocular SLAM for a small swarm of drones. Margarita has given invited keynotes at the World Economic Forum in Davos, TEDx, and ICRA, and she was featured in Robohub’s 2016 list of “25 women in Robotics you need to know about”. In 2023, she won the ERC Consolidator Grant to research advanced robotic perception.

View Details

The 2024 IEEE International Conference on Robotics and Automation (ICRA) will take place from 13-17 May, in Yokohama, Japan. The event will feature plenary and keynote talks, technical sessions, posters, workshops and tutorials.

Plenary speakersThere are three plenary talks at the conference this year:

• Yoky Matsuoka – How to Turn a Roboticist into a Corporate Explorer
• Sami Haddadin – The Great Robot Accelerator: Collective Learning of Optimal Embodied AI
• Sunil K Agrawal – Rehabilitation Robotics: How to Improve Daily Functions in People with Impairments?

Keynote talksThere will be 15 keynote talks, given by:

• Lianqing Liu – Biosyncretic sensing, actuation and intelligence for robotics
• Dawn M. Tilbury – Digital Twins for Manufacturing Automation
• Claudio Pacchierotti – Beyond Force Feedback: Cutaneous Haptics in Human-centered Robotics
• Yu Sun – Medical Robotics for Cell Surgery – Science and Applications
• Yasuhisa Hirata – Adaptable AI-enabled Robots to Create a Vibrant Society – Moonshot R&D Projects in Japan
• Calin Belta – Formal Methods for Safety-Critical Control
• Manuel Catalano – Robots in the Wild: From Research Labs to the Real World
• Harold Soh – Building Guidance Bridges with Generative Models for Robot Learning and Control
• Lorenzo Sabattini – Unleashing the power of many: decentralized control of multi-robot systems
• Myunghee Kim – Human-wearable robot co-adaptation
• Yoko Yamanishi – Emergent Functions of Electrically-induced Bubbles and Intra-cellular-Cybernetic Avatar
• Kensuke Harada – Robotic manipulation aiming for industrial applications
• Iolanda Leite – The Quest for Social Robot Autonomy
• Rong Xiong – Integration of Robotics and AI: Changes and Challenges
• Mariana Medina-Sánchez – Tiny Robots, Big Impact: Transforming Gynecological Care

TutorialsThe tutorials will be held on Monday 13 May and Friday 17 May.

• Tutorial on Ergodic Control
• Connected Robotics Platform for ROS Deployment in Real-world Network Settings
• How to manage fleets of robots with open source software
• Riemann and Gauss meet Asimov: 2nd Tutorial on Geometric Methods in Robot Learning, Optimization and Control
• Cloud and Fog Robotics: A Hands-on Tutorial with ROS2 and FogROS2
• Choreographic swarms: From scripting to emergent expressive behaviors to CONNECT humans and robots

WorkshopsThe workshops will also be held on Monday 13 May and Friday 17 May. There are 73 to choose from this year.

• Debates on the Future of Robotics Research
• Workshop on Field Robotics
• Continuum and Soft robotics for medical applications with rising stars on the stage
• Agile Robotics: From Perception to Dynamic Action
• 6th Workshop on Long-Term Human Motion Prediction
• 2nd HERMES Workshop: Multi-Robot Sensing & Perception in Extreme Environments
• Towards Collaborative Partners: Design, Shared Control, and Robot Learning for Physical Human-Robot Interaction
• Society of Avatar-Symbiosis through Social Field Experiments
• 3rd Workshop on Future of Construction: Lifelong Learning Robots in Changing Construction Sites
• What does Responsible Robotics mean?: Stretching roboticists’ horizons from an academic, government and philosophical perspective
• Workshop on Ontologies and Standards for Robotics and Automation
• Exploring Role Allocation in Human-Robot Co-Manipulation
• C4SR+: Continuum, Compliant, Cooperative, Cognitive Surgical Robotic Systems in the Embodied AI Era
• Dynamic Duos: Human-Robot Co-Worker Adaptation in Manufacturing
• Autonomy in Robotics Surgery: State of the art, technical and regulatory challenges for clinical application
• 2nd Robot-Assisted Medical Imaging ICRA-RAMI
• Back to the Future: Robot Learning Going Probabilistic
• Bioinspired, soft and other novel design paradigms for aerial robotics
• Impulsive motion in soft robotic and microrobotic systems
• Agile Movements II: Animal Behavior, Biomechanics, and Robot Devices
• Bimanual manipulation: On kitchen challenges
• The robotics, psychology and neuroscience of body augmentation
• Sustainable Soft Robots: Working with the Environment
• Robotics and Sustainability: A Bidirectional Relationship
• 2nd Workshop on Mobile Manipulation and Embodied Intelligence (MOMA) Integrating Perception, Learning and Control for Full Autonomy
• (Re)designing the tree of robotic life: a game of alternative timelines
• Advancing Sustainable Food Systems through Agri-Robotics Innovations
• Co-design in Robotics: Theory, Practice, and Challenges
• Robot Trust for Symbiotic Societies
• Soft Continuum Manipulators: Bottlenecks and Applications
• Advanced human-robot interfaces based on physiological signals and sensory stimulations
• Assistive Systems: Lab to Patient Care
• MAD-Games: Workshop on Multi-Agent Dynamic Games
• Workshop on Robot Ethics – Ethical, Legal and User Perspectives in Robotics and Automation (WOROBET)
• Applications and Future Directions of Affective Technologies
• ProxyTouch: Whole-body Proxy-Tactile Architectures for Industrial and Service Applications
• Bio-inspired robotics and robotics for biology
• A Future Roadmap for Sensorimotor Skill Learning for Robot Manipulation
• Robotics and Automation in Nuclear Environments
• 3D Visual Representations for Robot Manipulation
• Advancements in Trajectory Optimization and Model Predictive Control for Legged Systems – 2nd Edition
• Robots for Understanding Natural Ecosystems
• Cooking Robotics: Perception and motion planning
• Robot Software Architectures (RSA24)
• Multi-Object Grasping: Progress and Prospects
• First Workshop on Vision-Language Models for Navigation and Manipulation
• Radar in Robotics: Resilience from Signal to Navigation
• Anthropomorphic and zoomorphic end-effectors: asset or useless bias?
• Workshop on Resilient Off-road Autonomy
• Breaking Swarm Stereotypes
• Translational research in medical robotics: From Lab Bench to Clinical Use – How to?
• 4th Workshop on Representing and Manipulating Deformable Objects
• Loco-Manipulation: Algorithms, Challenges & Applications
• Cognition across species: from nature to robotic application
• Unconventional Robots: Universal Lessons for Designing Unique Systems
• Accelerating Discovery in Natural Science Laboratories with AI and Robotics
• Human-Robot Companionship for Healthcare and Wellness: Which Form of Companionship for What Type of Care?
• Humanoid Whole-body Control: From human motion understanding to humanoid locomotion
• RoboNerF: Neural Fields in Robotics
• ViTac 2024: Robot Embodiment through Visuo-Tactile Perception
• Supervised autonomy: how to shape human-robot interaction from the body to the brain
• Robots and roboticists in the age of climate change
• Human-Centric Multilateral Teleoperation: Perception, Telecommunication, and Coordination
• Innovations and Applications of Human Modeling in Physical Human-Robot Interaction
• How to Ensure Correct Robot Behaviors? Software Challenges in Formal Methods for Robotics
• Advancing Wearable Devices and Applications through Novel Design, Sensing, Actuation, and AI
• Wearable Intelligence for Healthcare Robotics (WIHR): from Brain Activity to Body Movements
• Emerging Technologies in Smart Exoskeleton Systems
• Expanding Frontiers of Sim2Real: Robotics, biomechanics, plasma physics, chip design, and beyond
• Nursing Robotics: a new field emerging from the integration between robotics and nursing science
• Speed-dating to long-term relationships: Art-robot Residencies Enabled by Common Language
• Workshop on Human-aligned Reinforcement Learning for Autonomous Agents and Robots
• 2nd Workshop on NeuroDesign in Human-Robot Interaction: The making of engaging HRI technology your brain can’t resist

You can see the programme overview here, with a detailed programme available here.

View Details

Suction cup grasping a stone – Image credit: Tianqi Yue

The team, based at Bristol Robotics Laboratory, studied the structures of octopus biological suckers, which have superb adaptive suction abilities enabling them to anchor to rock.

In their findings, published in the journal PNAS today, the researchers show how they were able create a multi-layer soft structure and an artificial fluidic system to mimic the musculature and mucus structures of biological suckers.

Suction is a highly evolved biological adhesion strategy for soft-body organisms to achieve strong grasping on various objects. Biological suckers can adaptively attach to dry complex surfaces such as rocks and shells, which are extremely challenging for current artificial suction cups. Although the adaptive suction of biological suckers is believed to be the result of their soft body’s mechanical deformation, some studies imply that in-sucker mucus secretion may be another critical factor in helping attach to complex surfaces, thanks to its high viscosity.

Lead author Tianqi Yue explained: “The most important development is that we successfully demonstrated the effectiveness of the combination of mechanical conformation – the use of soft materials to conform to surface shape, and liquid seal – the spread of water onto the contacting surface for improving the suction adaptability on complex surfaces. This may also be the secret behind biological organisms ability to achieve adaptive suction.”

Their multi-scale suction mechanism is an organic combination of mechanical conformation and regulated water seal. Multi-layer soft materials first generate a rough mechanical conformation to the substrate, reducing leaking apertures to just micrometres. The remaining micron-sized apertures are then sealed by regulated water secretion from an artificial fluidic system based on the physical model, thereby the suction cup achieves long suction longevity on diverse surfaces but with minimal overflow.

Tianqi added: “We believe the presented multi-scale adaptive suction mechanism is a powerful new adaptive suction strategy which may be instrumental in the development of versatile soft adhesion.

”Current industrial solutions use always-on air pumps to actively generate the suction however, these are noisy and waste energy.

“With no need for a pump, it is well known that many natural organisms with suckers, including octopuses, some fishes such as suckerfish and remoras, leeches, gastropods and echinoderms, can maintain their superb adaptive suction on complex surfaces by exploiting their soft body structures.”

The findings have great potential for industrial applications, such as providing a next-generation robotic gripper for grasping a variety of irregular objects.

The team now plan to build a more intelligent suction cup, by embedding sensors into the suction cup to regulate suction cup’s behaviour.

Paper

‘Bioinspired multiscale adaptive suction on complex dry surfaces enhanced by regulated water secretion’ by Tianqi Yue, Weiyong Si, Alex Keller, Chenguang Yang, Hermes Bloomfield-Gadêlha and Jonathan Rossiter in PNAS.

View Details

The Open Source Robotics Foundation (OSRF) is pleased to announce the creation of the Open Source Robotics Alliance (OSRA), a new initiative to strengthen the governance of our open-source robotics software projects and ensure the health of the Robot Operating System (ROS) Suite community for many years to come. The OSRA will use a mixed membership and meritocratic model, following other successful foundations for open-source projects, including The Linux Foundation and the Eclipse Foundation.

The OSRA is extending an open invitation to all community stakeholders to participate in the technical oversight, direction, development, and support of the OSRF’s open source projects – ROS, Gazebo, Open-RMF, and their infrastructure. Involvement across the robotics ecosystem is crucial to this initiative.

The center of the OSRA will be the Technical Governance Committee (TGC), which will oversee the activities of various Project Management Committees, Technical Committees, Special
Interest Groups, and Working Groups. As a charitable program of the OSRF, overall responsibility for the OSRA remains with the OSRF Board.

The Alliance has received early support for our vision from prominent organizations such as NVIDIA, our inaugural Platinum member.

“NVIDIA develops with ROS 2 to bring accelerated computing and AI to developers, researchers, and commercial applications,” said Gordon Grigor, VP Robotics Software, NVIDIA. “As an inaugural platinum member of OSRA, we will collaborate to advance open-source robotics throughout the ecosystem by aiding development efforts, and providing governance and continuity.”

Intrinsic also continues its support of Open Robotics with its inaugural Platinum membership. “From the numerous contributions made by our team at Intrinsic across projects like ROS, Gazebo, and Open-RMF as part of the Open Robotics community, to our acquisition of the Open Source Robotics Corporation (OSRC), we’ve invested deeply in the open source community, and we look forward to continuing our support of the ecosystem as an inaugural member of the OSRA,” said Wendy Tan White, CEO of Intrinsic. Qualcomm Technologies rounds out the inaugural Platinum membership group and signals its commitment to open source robotics as well.

“Qualcomm Technologies is excited to join the Open Source Robotics Alliance (OSRA) to help drive the development of open-source robotics software and foster the growth of the vibrant ROS Suite developer community,” stated Dev Singh, Vice President of Business Development and Head of Robotics & Industrial Automation at Qualcomm Technologies. “Leveraging our longstanding history in on-device AI processing and heterogeneous computing, our comprehensive family of robotics platforms addresses all types of robots to deliver the benefits of AI at the edge.”

NVIDIA, Intrinsic, and Qualcomm Technologies join nine other inaugural members at press time, including Gold members Apex.ai and Zettascale, Silver members Clearpath Robotics, Ekumen, eProsima, and PickNik, and Associate member Silicon Valley Robotics. Initial Supporting Organizations include Canonical and Open Navigation. Incoming members include Bosch and ROS-Industrial, and several others to be announced soon.

Membership applications are now open for organizations and individuals interested in joining the OSRA and supporting the future of open source robotics. Instructions on how to apply and information on the member levels and benefits are available at www.osralliance.org.

About Open Robotics: Open Robotics is the umbrella term for the Open Source Robotics Foundation (OSRF) and its initiatives. Founded in 2012, the OSRF is a California nonprofit public benefit corporation exempt under Section 501(c)(3) of the Internal Revenue Code. Its flagship open-source robotics software, ROS, is the world’s most widely adopted robotics framework suite. For more information about its new OSRA initiative, please visit www.osralliance.org.

View Details

Claire chatted to Patricia Shaw from Aberystwyth University all about home assistance robots, and robot learning and development.

Patricia Shaw is a Senior Lecturer in Computer Science and Robotics at Aberystwyth University. Her current research interests include technology for assistive living and she is currently leading on establishing a new Smart Home Lab at the university. This new lab will be used to research, develop and test a wide range of sensing technology for monitoring activities in the home as well as robots ranging from companions to assistants around the home. She strongly supports public engagement and has been coordinating regional robotics week events for the last 6 years.

View Details

Claire chatted to Rav Chunilal from Sellafield all about robotics and AI for nuclear decommissioning.

Rav Chunilal is the Head of Robotics & Artificial Intelligence at Sellafield Ltd, the UK’s largest nuclear site. He is responsible for developing and deploying Robotics and AI technologies to decommission the site safely, efficiently, and cost-effectively. Under Rav’s leadership, Sellafield Ltd and the Nuclear Decommissioning Authority Group has established itself as a leader in the use of Robotics and AI for nuclear decommissioning. The company has developed and deployed a wide range of robotic technologies to perform tasks such as remote inspection, waste handling, and demolition.

View Details

Thanks to those that sent and suggested AI and robotics-themed holiday videos, images, and stories. Here’s a sample to get you into the spirit this season.

Josh Miller used ChatGPT to create the 12 days of Christmas. Prompt: “I want to make 12 images in an early 1980s era to depict the twelve days of Christmas”.

See the pictures in higher definition here.

“Santa’s Secret: Robots Saved Christmas! | CPS XMAS video 2023” from Chair of Cyber-Physical Systems.

“AI rewriting classic Christmas stories” from GingerPale.

Using PaLM2 to generate haikus.

Prompt: “Write me a haiku about an alternative xmas”.
No gifts, no grand feast,
Just laughter and joy, warmth spread,
Kindness fills the air.

Prompt: “Write a haiku about the winter holiday season”.
Snow blankets the land,
Hearts aglow with love’s warmth,
Joyous cheer surrounds.

“Put a Bow on It | Happy Holidays” from Boston Dynamics.

“Christmas Magic: Animated Backgrounds for an Enchanting Atmosphere” from AI-ArtificialIntelligence.

“XMAS – Episode 3: AI – Christmas music for the HAKOM Christmas party” from HAKOM Time Series.

“Christmas tree in python notebook with Turtle” from Learn Upwards.

“FZI Living Lab Christmas Robotics 2023 – Robotic Reindeer Rescue” from FZI channel.

A letter to Santa generated by PaLM2. Prompt: “Write me a letter to Santa Claus. I don’t want any presents, just inspiration in my machine learning research for 2024.”

Dear Santa Claus,

As the year comes to a close, I find myself reflecting on the remarkable advancements in machine learning that have transformed our world. From self-driving cars to personalized medicine, these innovations have opened up new possibilities and enriched our lives in ways we could have only imagined a few decades ago.

I am writing to you today not to ask for material gifts, but rather for something more precious: inspiration to continue exploring the frontiers of machine learning in the year to come. As a researcher in this field, I am eager to contribute to the development of algorithms and techniques that can further enhance our understanding of the world around us and create solutions to pressing global challenges.

I am particularly fascinated by the potential of machine learning to address the climate crisis. By harnessing the power of data and computation, we can develop more efficient renewable energy sources, optimize transportation systems, and mitigate the impact of extreme weather events. Additionally, I believe that machine learning can play a crucial role in improving education, healthcare, and environmental sustainability.

I know that you have a special ability to see the world in a unique and insightful way. I hope that you can share some of your wisdom with me and guide me towards promising research directions. I am eager to embark on new challenges and make a positive impact on the world through machine learning.

Thank you for considering my request, Santa Claus. I look forward to your guidance and inspiration as I pursue my research endeavors in the year ahead.

With warmest regards,

[Your Name]

View Details

By Farshad Arvin, Martin Stefanec, and Tomas Krajnik

Be it the news or the dwindling number of creatures hitting your windscreens, it will not have evaded you that the insect world in bad shape.

In the last three decades, the global biomass of flying insects has shrunk by 75%. Among the trend’s most notables victims is the world’s most important pollinator, the honeybee. In the United States, 48% of honeybee colonies died in 2023 alone, making it the second deadliest year on record. This significant loss is due in part to colony collapse disorder (CCD), the sudden disappearance of bees. In contrast, European countries report lower but still worrisome rates of colony losses, ranging from 6% to 32%.

This decline causes many of our essential food crops to be under-pollinated, a phenomenon that threatens our society’s food security.

Debunking the sci-fi myth of robotic beesSo, what can be done? Given pesticides’ role in the decline of bee colonies, commonly proposed solutions include a shift away from industrial farming and toward less pesticide-intensive, more sustainable forms of agriculture.

Others tend to look toward the sci-fi end of things, with some scientists imagining that we could eventually replace live honeybees with robotic ones. Such artificial bees could interact with flowers like natural insects, maintaining pollination levels despite the declining numbers of natural pollinators. The vision of artificial pollinators contributed to ingenious designs of insect-sized robots capable of flying.

In reality, such inventions are more effective at educating us over engineers’ fantasies than they are at reviving bee colonies, so slim are their prospects of materialising. First, these artificial pollinators would have to be equipped for much more more than just flying. Daily tasks carried out by the common bee include searching for plants, identifying flowers, unobtrusively interacting with them, locating energy sources, ducking potential predators, and dealing with adverse weather conditions. Robots would have to perform all of these in the wild with a very high degree of reliability since any broken-down or lost robot can cause damage and spread pollution. Second, it remains to be seen whether our technological knowledge would be even capable of manufacturing such inventions. This is without even mentioning the price tag of a swarm of robots capable of substituting pollination provided by a single honeybee colony.

Inside a smart hive Bees on one of Hiveopolis’s augmented hives.
Hiveopolis, Fourni par l’auteur
Rather than trying to replace honeybees with robots, our two latest projects funded by the European Union propose that the robots and honeybees actually team up. Were these to succeed, struggling honeybee colonies could be transformed into bio-hybrid entities consisting of biological and technological components with complementary skills. This would hopefully boost and secure the colonies’ population growth as more bees survive over harsh winters and yield more foragers to pollinate surrounding ecosystems.

The first of these projects, Hiveopolis, investigates how the complex decentralised decision-making mechanism in a honeybee colony can be nudged by digital technology. Begun in 2019 and set to end in March 2024, the experiment introduces technology into three observation hives each containing 4,000 bees, by contrast to 40,000 bees for a normal colony.

The foundation of an augmented honeycomb.
Hiveopolis, Fourni par l’auteur
Within this honeybee smart home, combs have integrated temperature sensors and heating devices, allowing the bees to enjoy optimal conditions inside the colony. Since bees tend to snuggle up to warmer locations, the combs also enables us to direct them toward different areas of the hive. And as if that control weren’t enough, the hives are also equipped with a system of electronic gates that monitors the insects movements. Both technologies allow us to decide where the bees store honey and pollen, but also when they vacate the combs so as to enable us to harvest honey. Last but not least, the smart hive contains a robotic dancing bee that can direct foraging bees toward areas with plants to be pollinated.

Due to the experiment’s small scale, it is impossible to draw conclusions on the extent to which our technologies may have prevented bee losses. However, there is little doubt what we have seen thus far give reasons to be hopeful. We can confidently assert that our smart beehives allowed colonies to survive extreme cold during the winter in a way that wouldn’t otherwise be possible. To precisely assess how many bees these technologies have saved would require upscaling the experiment to hundreds of colonies.

Pampering the queen beeOur second EU-funded project, RoboRoyale, focuses on the honeybee queen and her courtyard bees, with robots in this instance continuously monitoring and interacting with her Royal Highness.

Come 2024, we will equip each hive with a group of six bee-sized robots, which will groom and feed the honeybee queen to affect the number of eggs she lays. Some of these robots will be equipped with royal jelly micro-pumps to feed her, while others will feature compliant micro-actuators to groom her. These robots will then be connected to a larger robotic arm with infrared cameras, that will continuously monitor the queen and her vicinity.

A RoboRoyale robot arm susses out a honeybee colony.
RoboRoyale, Fourni par l’auteur
As witnessed by the photo to the right and also below, we have already been able to successfully introduce the robotic arm within a living colony. There it continuously monitored the queen and determined her whereabouts through light stimuli.

Emulating the worker beesIn a second phase, it is hoped the bee-sized robots and robotic arm will be able to emulate the behaviour of the workers, the female bees lacking reproductive capacity who attend to the queen and feed her royal jelly. Rich in water, proteins, carbohydrates, lipids, vitamins and minerals, this nutritious substance secreted by the glands of the worker bees enables the queen to lay up to thousands of eggs a day.

Worker bees also engage in cleaning the queen, which involves licking her. During such interactions, they collect some of the queen’s pheromones and disperse them throughout the colony as they move across the hive. The presence of these pheromones controls many of the colony’s behaviours and notifies the colony of a queen’s presence. For example, in the event of the queen’s demise, a new queen must be quickly reared from an egg laid by the late queen, leaving only a narrow time window for the colony to react.

One of RoboRoyale’s first experiments has consisted in simple interactions with the queen bee through light stimulus. The next months will then see the robotic arm stretch out to physically touch and groom her.
RoboRoyale, Fourni par l’auteur
Finally, it is believed worker bees may also act as the queen’s guides, leading her to laying eggs in specific comb cells. The size of these cells can determine if the queen lays a diploid or haploid egg, resulting in the bee developing into either into drone (male) or worker (female) bee. Taking over these guiding duties could affect no less than the rate’s entire reproductive rate.

How robots can prevent bee cannibalismThis could have another virtuous effect: preventing cannibalism.

During tough times, such as long periods of rain, bees have to make do with little pollen intake. This forces them to feed young larvae to older ones so that at least the older larvae has a chance to survive. Through RoboRoyale, we will look not only to reduce chances of this behaviour occurring, but also quantify to what extent it occurs under normal conditions.

Ultimately, our robots will enable us to deepen our understanding of the very complex regulation processes inside honeybee colonies through novel experimental procedures. The insights gained from these new research tracks will be necessary to better protect these valuable social insects and ensure sufficient pollination in the future – a high stakes enterprise for food security.

This article is the result of The Conversation’s collaboration with Horizon, the EU research and innovation magazine.

Farshad Arvin is a member of the Department of Computer Science at Durham University in the UK. The research of Farshad Arvin is primarily funded by the EU H2020 and Horizon Europe programmes.

Martin Stefanec is a member of the Institute of Biology at the University of Graz. He has received funding from the EU programs H2020 and Horizon Europe.

Tomas Krajnik is member of the Institute of Electrical and Electronics Engineers (IEEE). The research of Tomas Krajnik is primarily funded by EU H2020 Horizon programme and Czech National Science Foundation.

View Details

Claire chatted to Ayse Kucukyilmaz from the University of Nottingham about collaboration, conflict and failure in human-robot interactions.

Ayse Kucukyilmaz is Assistant Professor in the School of Computer Science at the University of Nottingham and a founding member of the CHART research group. Her research focus is human-centered robotics, where she specialises in haptic shared and traded control for physical human-robot interaction. Her group primarily works on adjustable autonomy paradigms to enable dynamic switching behaviours between different levels of robotic autonomy (e.g. full human control vs. full autonomy) during shared control of a physical task, enabling effective human-autonomy teaming.

View Details

Congratulations to Dautzenberg Roman and his team of researchers, who won the IROS 2023 Best Paper Award on Mobile Manipulation sponsored by OMRON Sinic X Corp. for their paper “A perching and tilting aerial robot for precise and versatile power tool work on vertical walls“. Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper?Our paper shows a an aerial robot (think “drone”) which can exert large forces in the horizontal direction, i.e. onto walls. This is a difficult task, as UAVs usually rely on thrust vectoring to apply horizontal forces and thus can only apply small forces before losing control authority. By perching onto walls, our system no longer needs the propulsion to remain at a desired site. Instead we use the propellers to achieve large reaction forces in any direction, also onto walls! Additionally, perching allows extreme precision, as the tool can be moved and re-adjusted, as well as being unaffected by external disturbances such as gusts of wind.

Could you tell us about the implications of your research and why it is an interesting area for study?Precision, force exertion and mobility are the three (of many) criteria where robots – and those that develop them – make trade-offs. Our research shows that the system we designed can exert large forces precisely with only minimal compromises on mobility. This widens the horizon of conceivable tasks for aerial robots, as well as serving as the next link in automating the chain of tasks need to perform many procedures on construction sites, or on remote, complex or hazardous environments.

Could you explain your methodology?The main aim of our paper is to characterize the behavior and performance of the system, and comparing the system to other aerial robots. To achieve this, we investigated the perching and tool positioning accuracy, as well as comparing the applicable reaction forces with other systems.

Further, the paper shows the power consumption and rotational velocities of the propellers for the various phases of a typical operation, as well as how certain mechanism of the aerial robot are configured. This allows for a deeper understanding of the characteristics of the aerial robot.

What were your main findings?Most notably, we show the perching precision to be within +-10cm of a desired location over 30 consecutive attempts and tool positioning to have mm-level accuracy even in a “worst-case” scenario. Power consumption while perching on typical concrete is extremely low and the system is capable of performing various tasks (drilling, screwing) also in quasi-realistic, outdoor scenarios.

What further work are you planning in this area?Going forward, enhancing the capabilities will be a priority. This relates both to the types of surface manipulations that can be performed, but also the surfaces onto which the system can perch.

About the author

| | Dautzenberg Roman is currently a Masters student at ETH Zürich and Team Leader at AITHON. AITHON is a research project which is transforming into a start-up for aerial construction robotics. They are a core team of 8 engineers, working under the guidance of the Autonomous Systems Lab at ETH Zürich and located at the Innovation Park Switzerland in Dübendorf. |

View Details

Claire chatted to Jorvon (Odd-Jayy) Moss from Digikey about making robots at home, and robot design and aesthetics.

Commonly known as Odd-Jayy, Jorvon Moss is an accomplished Maker best known for his Robotic Oddities. Jayy’s art background, BFA in Illustration, and self-taught electronics skills have combined to help launch his career and promote the wonderful world of STEAM (Science, Technology, Engineering, Art, Math). This achievement, and the many viral videos under his belt, gained him recognition from major forces in the industry; including Digi-Key Electronics, Tested Inc. with Adam Savage, various electronic and tech Faires, and as the first Black person in Make Magazine.

View Details

I came to the Silicon Valley region in 2010 because I knew it was the robotics center of the world, but it certainly doesn’t get anywhere near the media attention that some other robotics regions do. In California, robotics technology is a small fish in a much bigger technology pond, and that tends to conceal how important Californian companies are to the robotics revolution.

This conservative dataset from Pitchbook [Vertical: Robotics and Drones] provides data for 7166 robotics and drones companies, although a more customized search would provide closer to 10,000 robotics companies world wide. Regions ordered by size are:

• North America 2802
• Asia 2337
• Europe 2285
• Middle East 321
• Oceania 155
• South America 111
• Africa 63
• Central America 13

USA robotics companies by state1. California = 843 (667) * no of companies followed by no of head quarters 2. Texas = 220 (159) 3. New York = 193 (121) 4. Massachusetts = 191 (135) 5. Florida = 136 (95) 6. Pennsylvania = 113 (89) 7. Washington = 85 (61) 8. Colorado = 83 (57) 9. Virginia = 81 (61) 10. Michigan = 70 (56) 11. Illinois = 66 (43) 12. Ohio = 65 (56) 13. Georgia = 64 (46) 14. New Jersey = 53 (36) 15. Delaware = 49 (18) 16. Maryland = 48 (34) 17. Arizona = 48 (37) 18. Nevada = 42 (29) 19. North Carolina = 39 (29) 20. Minnesota = 31 (25) 21. Utah = 30 (24) 22. Indiana = 29 (26) 23. Oregon = 29 (20) 24. Connecticut = 27 (22) 25. DC = 26 (12) 26. Alabama = 25 (21) 27. Tennessee = 20 (18) 28. Iowa = 17 (14) 29. New Mexico = 17 (15) 30. Missouri = 17 (16) 31. Wisconsin = 15 (12) 32. North Dakota = 14 (8) 33. South Carolina = 13 (11) 34. New Hampshire = 13 (12) 35. Nebraska = 13 (11) 36. Oklahoma = 10 (8) 37. Kentucky = 10 (7) 38. Kansas = 9 (9) 39. Louisiana = 9 (8) 40. Rhode Island = 8 (6) 41. Idaho = 8 (6) 42. Maine = 5 (5) 43. Montana = 5 (4) 44. Wyoming = 5 (3) 45. Mississippi = 3 (1) 46. Arkansas = 3 (2) 47. Alaska = 3 (3) 48. Hawaii = 2 (1) 49. West Virginia = 1 (1) 50. South Dakota = 1 (0)

Note – this number in brackets is for HQ locations, whereas the first number is for all company locations. The end results and rankings are practically the same.

ASIA robotics companies by country1. China = 1350 2. Japan = 283 3. India = 261 4. South Korea = 246 5. Israel = 193 6. Hong Kong = 72 7. Russia = 69 8. United Arab Emirates = 50 9. Turkey = 48 10. Malaysia = 35 11. Taiwan = 21 12. Saudi Arabia = 19 13. Thailand = 13 14. Vietnam = 12 15. Indonesia = 10 16. Lebanon = 7 17. Kazakhstan = 3 18. Iran = 3 19. Kuwait = 3 20. Oman = 3 21. Qatar = 3 22. Pakistan = 3 23. Philippines = 2 24. Bahrain = 2 25. Georgia = 2 26. Sri Lanka = 2 27. Azerbaijan = 1 28. Nepal = 1 29. Armenia = 1 30. Burma/Myanmar = 1

Countries with no robotics; Yemen, Iraq, Syria, Turkmenistan, Afghanistan, Syria, Jordan, Uzbekistan, Kyrgyzstan, Tajikistan, Bangladesh, Bhutan, Mongolia, Cambodia, Laos, North Korea, East Timor.

UK/EUROPE robotics companies by country1. United Kingdom = 443 2. Germany = 331 3. France = 320 4. Spain = 159 5. Netherlands = 156 6. Switzerland = 140 7. Italy = 125 8. Denmark = 115 9. Sweden = 85 10. Norway = 80 11. Poland = 74 12. Belgium = 72 13. Russia = 69 14. Austria = 51 15. Turkey = 48 16. Finland = 45 17. Portugal = 36 18. Ireland = 28 19. Estonia = 24 20. Ukraine = 22 21. Czech Republic = 19 22. Romania = 19 23. Hungary = 18 24. Lithuania = 18 25. Latvia = 15 26. Greece = 15 27. Bulgaria = 11 28. Slovakia = 10 29. Croatia = 7 30. Slovenia = 6 31. Serbia = 6 32. Belarus = 4 33. Iceland = 3 34. Cyprus = 2 35. Bosnia & Herzegovina = 1

Countries with no robotics; Andorra, Montenegro, Albania, Macedonia, Kosovo, Moldova, Malta, Vatican City.

CANADA robotics companies by region1. Ontario = 144 2. British Colombia = 60 3. Quebec = 53 4. Alberta = 34 5. Manitoba = 7 6. Saskatchewan = 6 7. Newfoundland & Labrador = 2 8. Yukon = 1

Regions with no robotics; Nunavut, Northwest Territories.

View Details

Claire chatted to Masoumeh (Iran) Mansouri from the University of Birmingham about culturally sensitive robots and planning in complex environments.

Masoumeh Mansouri is an Associate Professor in the School of Computer Science at the University of Birmingham. Her research includes two complementary areas: (i) developing hybrid robot planning methods for unstructured environments shared with humans, and (ii) exploring topics at the intersection of cultural theories and robotics. In the latter, her main goal is to study whether/how robots can be culturally sensitive given the broad definitions of culture in different fields of study.

View Details

If you look at the UN Sustainable Development Goals, it’s clear that robots have a huge role to play in advancing the SDGs. However the field of Sustainable Robotics is more than just the application area. For every application that robotics can improve in sustainability, you have to also address the question – what are the additional costs or benefits all the way along the supply chain. What are the ‘externalities’, or additional costs/benefits, of using robots to solve the problem. Does the use of robotics bring a decrease or an increase to:

• power costs
• production costs
• labor costs
• supply chain costs
• supply chain mileage
• raw materials consumption
• and raw material choice

Solving our economic and environmental global challenges should not involve adding to the existing problems or creating new ones. So it’s important that we look beyond the first order ways in which robotics can solve global sustainable development goals and address every level at which robotics can have an impact.

Here I propose 5 levels of sustainability to frame the discussion, much as the 5 levels of autonomy have helped define the stages of autonomous mobility.

Level 1: Robots for existing recyclingLevel 1 of Sustainable Robotics is simply making existing processes in sustainability more efficient, affordable and deployable. Making recycling better. Companies that are great examples are: AMP Robotics, Recycleye, MachineEx, Pellenc ST, Greyparrot, Everlast Labs and Fanuc. Here’s an explainer video from Fanuc.

“Because of AI, because of the robotic arms, we have seen plants recover 10, 20, 30% more than what they have been doing previously,” said JD Ambati, CEO of EverestLabs. “They have been losing millions of dollars to the landfill, and because of AI, they were able to identify the value of the losses and deploy robotic arms to capture that.”

Some other examples of Level 1 use robots to better monitor aquaculture, or robots to clean or install solar farms and wind turbines. If the robotics technology improves existing recycling practices then it is at Level 1 of Sustainable Robotics.

Level 2: Robots enabling new recyclingLevel 2 of Sustainable Robotics is where robotics allows new materials to be recycled and in new industry application areas. A great example of this is Urban Machines, which salvages timber from construction sites and transforms it back into useable materials, something that was too difficult to do at any scale previously.

Construction using onsite materials and robotics 3D printing is another example, as seen in the NASA Habitat Challenge, sponsored by Caterpillar, Bechtel and Brick & Mortar Ventures.

Some other examples are the ocean or lake going garbage collecting robots like Waste Shark from Ran Marine, River Cleaning, or Searial Cleaners, a Quebec company whose robots were deployed in the Great Lakes Plastic Cleanup, helping to remove 74,000 plastic pieces from four lakes since 2020.

Searial Cleaners is angling for its BeBot and PixieDrone to be used as janitorial tools for beaches, marinas and golf courses, and the BeBot offers ample room for company branding. The equipment emerged from the mission of the Great Lakes Plastic Cleanup (GLPC) to harness new technologies against litter. The program also uses other devices including the Seabin, which sits in water and sucks in trash, and the Enviropod LittaTrap filter for stormwater drains.

If it’s a brand new way to practice recycling with robotic technology, then it’s at Level 2 of Sustainable Robotics.

Level 3: Robots electrifying everythingOne of the biggest sustainability shifts enabled by robotics is the transition from fossil fuel powered transport, logistics and agricultural machinery into BEV, or Battery Electric Vehicle technology. On top of radically reducing emissions, the increasing use of smaller autonomous electric vehicles across first, last and middle mile can change the total number of trips taken, as well as reducing the need for large vehicles that are partially loaded taking longer trips.

Monarch Tractor’s MK-V is the world’s first electric tractor, and is ‘driver optional’, meaning it can be driven or operate autonomously, providing greater flexibility for farmers. Of course the increased use of computer vision and AI across all agrobots increase sustainability, by enabling precision or regenerative agriculture with less need for chemical solutions. Technically, these improvements to agricultural practice are Level 2 of Sustainable Robotics.

However, the use of smaller sized fully autonomous agricultural robots, such as Meropy, Burro.ai, SwarmFarm, Muddy Machines and Small Robot Company also reduces the size and soil compaction associated with agricultural machinery, and make it possible to tend smaller strip farms by machine. This is Level 3 of Sustainable Robotics.

Level 4: RobotsThe higher the sustainability level, the deeper it is into the actual design and construction of the robot system. Switching to electric from fossil fuels is a small step. Switching to locally sourced or produced materials is another. Switching to recyclable materials is another step towards fully sustainable robotics.

OhmniLabs utilize 3D printing in their robot construction, allowing them to export robots to 47 countries, while also manufacturing locally in Silicon Valley.

Meanwhile, Cornell researchers Wendy Ju and Ilan Mandel have introduced the phrase ‘Garbatrage’ to describe the opportunity to prototype or build robots using components recycled from other consumer electronics, like these hoverboards.

“The time is ripe for a practice like garbatrage, both for sustainability reasons and considering the global supply shortages and international trade issues of the last few years,” the researchers said.

This is a great example of Level 4 of Sustainable Robotics.

Level 5: Self-powering/repairing RobotsSelf powering or self repairing or self recycling robots are the Level 5 of Sustainable Robotics. In research, there are solutions like MilliMobile: A battery-free autonomous robot capable of operating on harvested solar and RF power. MilliMobile, developed at the Paul G. Allen School of Computer Science & Engineering, is the size of a penny and can steer itself, sense its environment, and communicate wirelessly using energy harvested from light and radio waves.

It’s not just research though. In the last two years, a number of solar powered agricultural robots have entered the market. Solinftec has a solar powered spray robot, as has EcoRobotix and AIGEN, which is also powered by wind.

Modular robotics will reduce our material wastage and energy needs by making robotics multipurpose, rather than requiring multiple specialist robots. Meanwhile self powering and self repairing technologies will allow robots to enter many previously unreachable areas, including off planet, while removing our reliance on the grid. As robots incorporate self repairing materials, the product lifecycle is increased. This is Level 5 of Sustainable Robotics.

And in the future?While we’re waiting for the future, here are a couple of resources for turning your entire company into a sustainable robotics company. Sustainable Manufacturing 101 from ITA, the International Trade Administration and the Sustainable Manufacturing Toolkit from the OECD.

References1. https://www.cnbc.com/2023/08/08/everestlabs-using-robotic-arms-and-ai-to-make-recycling-more-efficient.html 2. https://www.greenbiz.com/article/great-lakes-are-awash-plastic-can-robots-and-drones-help 3. https://www.economist.com/science-and-technology/2020/02/06/using-artificial-intelligence-agricultural-robots-are-on-the-rise 4. https://www.wired.co.uk/article/farming-robots-small-robot-company-tractors 5. https://news.cornell.edu/stories/2023/09/garbatrage-spins-e-waste-prototyping-gold

View Details

Feature Fields for Robotic Manipulation (F3RM) enables robots to interpret open-ended text prompts using natural language, helping the machines manipulate unfamiliar objects. The system’s 3D feature fields could be helpful in environments that contain thousands of objects, such as warehouses. Images courtesy of the researchers.

By Alex Shipps | MIT CSAIL

Imagine you’re visiting a friend abroad, and you look inside their fridge to see what would make for a great breakfast. Many of the items initially appear foreign to you, with each one encased in unfamiliar packaging and containers. Despite these visual distinctions, you begin to understand what each one is used for and pick them up as needed.

Inspired by humans’ ability to handle unfamiliar objects, a group from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) designed Feature Fields for Robotic Manipulation (F3RM), a system that blends 2D images with foundation model features into 3D scenes to help robots identify and grasp nearby items. F3RM can interpret open-ended language prompts from humans, making the method helpful in real-world environments that contain thousands of objects, like warehouses and households.

F3RM offers robots the ability to interpret open-ended text prompts using natural language, helping the machines manipulate objects. As a result, the machines can understand less-specific requests from humans and still complete the desired task. For example, if a user asks the robot to “pick up a tall mug,” the robot can locate and grab the item that best fits that description.

“Making robots that can actually generalize in the real world is incredibly hard,” says Ge Yang, postdoc at the National Science Foundation AI Institute for Artificial Intelligence and Fundamental Interactions and MIT CSAIL. “We really want to figure out how to do that, so with this project, we try to push for an aggressive level of generalization, from just three or four objects to anything we find in MIT’s Stata Center. We wanted to learn how to make robots as flexible as ourselves, since we can grasp and place objects even though we’ve never seen them before.”

Learning “what’s where by looking”The method could assist robots with picking items in large fulfillment centers with inevitable clutter and unpredictability. In these warehouses, robots are often given a description of the inventory that they’re required to identify. The robots must match the text provided to an object, regardless of variations in packaging, so that customers’ orders are shipped correctly.

For example, the fulfillment centers of major online retailers can contain millions of items, many of which a robot will have never encountered before. To operate at such a scale, robots need to understand the geometry and semantics of different items, with some being in tight spaces. With F3RM’s advanced spatial and semantic perception abilities, a robot could become more effective at locating an object, placing it in a bin, and then sending it along for packaging. Ultimately, this would help factory workers ship customers’ orders more efficiently.

“One thing that often surprises people with F3RM is that the same system also works on a room and building scale, and can be used to build simulation environments for robot learning and large maps,” says Yang. “But before we scale up this work further, we want to first make this system work really fast. This way, we can use this type of representation for more dynamic robotic control tasks, hopefully in real-time, so that robots that handle more dynamic tasks can use it for perception.”

The MIT team notes that F3RM’s ability to understand different scenes could make it useful in urban and household environments. For example, the approach could help personalized robots identify and pick up specific items. The system aids robots in grasping their surroundings — both physically and perceptively.

“Visual perception was defined by David Marr as the problem of knowing ‘what is where by looking,’” says senior author Phillip Isola, MIT associate professor of electrical engineering and computer science and CSAIL principal investigator. “Recent foundation models have gotten really good at knowing what they are looking at; they can recognize thousands of object categories and provide detailed text descriptions of images. At the same time, radiance fields have gotten really good at representing where stuff is in a scene. The combination of these two approaches can create a representation of what is where in 3D, and what our work shows is that this combination is especially useful for robotic tasks, which require manipulating objects in 3D.”

Creating a “digital twin”F3RM begins to understand its surroundings by taking pictures on a selfie stick. The mounted camera snaps 50 images at different poses, enabling it to build a neural radiance field (NeRF), a deep learning method that takes 2D images to construct a 3D scene. This collage of RGB photos creates a “digital twin” of its surroundings in the form of a 360-degree representation of what’s nearby.

In addition to a highly detailed neural radiance field, F3RM also builds a feature field to augment geometry with semantic information. The system uses CLIP, a vision foundation model trained on hundreds of millions of images to efficiently learn visual concepts. By reconstructing the 2D CLIP features for the images taken by the selfie stick, F3RM effectively lifts the 2D features into a 3D representation.

Keeping things open-endedAfter receiving a few demonstrations, the robot applies what it knows about geometry and semantics to grasp objects it has never encountered before. Once a user submits a text query, the robot searches through the space of possible grasps to identify those most likely to succeed in picking up the object requested by the user. Each potential option is scored based on its relevance to the prompt, similarity to the demonstrations the robot has been trained on, and if it causes any collisions. The highest-scored grasp is then chosen and executed.

To demonstrate the system’s ability to interpret open-ended requests from humans, the researchers prompted the robot to pick up Baymax, a character from Disney’s “Big Hero 6.” While F3RM had never been directly trained to pick up a toy of the cartoon superhero, the robot used its spatial awareness and vision-language features from the foundation models to decide which object to grasp and how to pick it up.

F3RM also enables users to specify which object they want the robot to handle at different levels of linguistic detail. For example, if there is a metal mug and a glass mug, the user can ask the robot for the “glass mug.” If the bot sees two glass mugs and one of them is filled with coffee and the other with juice, the user can ask for the “glass mug with coffee.” The foundation model features embedded within the feature field enable this level of open-ended understanding.

“If I showed a person how to pick up a mug by the lip, they could easily transfer that knowledge to pick up objects with similar geometries such as bowls, measuring beakers, or even rolls of tape. For robots, achieving this level of adaptability has been quite challenging,” says MIT PhD student, CSAIL affiliate, and co-lead author William Shen. “F3RM combines geometric understanding with semantics from foundation models trained on internet-scale data to enable this level of aggressive generalization from just a small number of demonstrations.”

Shen and Yang wrote the paper under the supervision of Isola, with MIT professor and CSAIL principal investigator Leslie Pack Kaelbling and undergraduate students Alan Yu and Jansen Wong as co-authors. The team was supported, in part, by Amazon.com Services, the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research’s Multidisciplinary University Initiative, the Army Research Office, the MIT-IBM Watson Lab, and the MIT Quest for Intelligence. Their work will be presented at the 2023 Conference on Robot Learning.

• PAPER – Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation.
  William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, and Phillip Isola. arXiv

View Details

Claire chatted to Carl Strathearn from Edinburgh Napier University about humanoid robots, realistic robot faces and speech.

Carl Strathearn is a researcher interested in creating assistive social humanoid robots with embodied AI systems that appear, function, and interact like humans. He believes that creating realistic humanoid robots is significant to humanity as the human face is the most natural interface for human communication, and by emulating these conditions, we can increase accessibility to state-of-the-art technology for everyone and support people with specific health conditions and circumstances in their day-to-day lives.

View Details

Anton Grabolle / Better Images of AI / Human-AI collaboration / Licenced by CC-BY 4.0

The AAAI Fall Symposium Series took place in Arlington, USA, and comprised seven different symposia. One of these, the tenth Artificial Intelligence for Human-Robot Interaction (AI-HRI) symposium was run as a hybrid in-person/online event, and we tuned in to the opening keynote, which was given by Patrícia Alves-Oliveira.

As a psychology student, Patrícia’s dream was to become a therapist. However, an internship, where she encountered a robot for the first time, inspired her to change her plans, and she decided to go into the field of human-robot interaction. Following a PhD in the field, she worked as a postdoc, before heading to industry as a designer in the Amazon Astro robot team.

Patrícia has worked on a number of interesting projects during her time in academia and in industry. Thinking about how to design robots for specific user needs, and keeping the user at the forefront during the design process, has been core to her work. She began by summarising three very different academic projects.

Creativity and roboticsThe objective of this project was to design, fabricate, and evaluate robots as creativity-provoking tools for kids. Patrícia created a social robot named YOLO (or Your Own Living Object) that she designed to be child-proof (in other words, it could withstand being dropped and knocked over), with the aim of trying to help children explore their creativity during play. A machine learning algorithm learns the pattern of play that the child has and adapts the robot behaviour accordingly. You can see the robot in action in the demo below:

FLEXI robotAs a postdoc project, Patrícia worked on building FLEXI, a social robot embodiment kit. This kit consists of a robot (with a face, and a torso with a screen on the front), which can be customised, and an open-source end-user programming interface designed to be user-friendly. The customisation element means that it can be used for many applications. The team has deployed FLEXI across three application scenarios: community-support, mental health, and education, with the aim of assessing the flexibility of the system. You can see the robot in action, in different scenarios, here.

Social diningThis project centred on a robotic arm for people with impaired mobility. Such systems already exist for assisting people with tasks such as eating. However, in a social context they can often form a barrier between the user and the rest of the group. The idea behind this project was to consider how such a robot could be adapted to work well in a social context, for example, during a meal with family or friends. The team interviewed people with impaired mobility to assess their needs, and came up with a set of design principles for creating robot-assisted feeding systems and an implementation guide for future research in this area. You can read the research paper on this project here.

You can find out more about these three projects, and the other projects that Patrícia has been involved in, here.

Astro robotPatrícia has long been interested in robots for the real world, and how this real-world experience is aligned with the study of robots in academia and industry. She decided to leave academia and join the Astro robot programme, which she felt was a great opportunity to work on a large-scale real-world robot project.

The Astro robot is a home robot designed to assist with tasks such as monitoring your house, delivering small objects within the home, recognising your pet, telling a story, or playing games.

Patrícia took us through a typical day in the life of a designer where she always has in mind the bigger picture of what the team is aiming for, in other words, what the ideal robot, and its interactions with humans, would look like. Coupled to that, the process is governed by core design tenets, such as the customer needs, and non-negotiable core elements that the robot should include. When considering a particular element of the robot design, for example, the delivery of an item in the robot tray, Patrícia uses storyboards to map out details of potential human-robot interactions. An important aspect of design concerns edge cases, which occur regularly in the real world. Good design will consider potential edge cases and incorporate ways to deal with them.

Patrícia closed by emphasising the importance of teamwork in the design process, in particular, the need for interdisciplinary teams; by considering design from many different points of view, the chance of innovation is higher.

You can find out more about the Artificial Intelligence for Human-Robot Interaction (AI-HRI) symposium here.

View Details

A science fiction/science fact review of Three Miles Down by Harry Turtledove, the fictionalized version of the Hughes Glomar Explorer expedition 50 years before the OceanGate Titan tragedy.

My new science fiction/science fact article for Science Robotics is out on why deep ocean robotics is hard. Especially when trying to bring up a sunken submarine 3 miles underwater, which the CIA actually did in 1974. It’s even harder if you’re trying to bring up an alien spaceship- which is the plot of Harry Turtledove’s new sci-fi novel Three Miles Under. It’s a delightful Forrest Gump version of that 1974 Hughes Glomar Explorer expedition. Though the expedition was 50 years before the OceanGate Titan tragedy, the same challenges exist for today’s robots. The robotics science in the book is very real, the aliens, not so much.

In 1974, the CIA deployed a 3 mile long, 6 million pound robot manipulator to recover a Russian submarine. The cover story was that Howard Hughes was deep sea mining for manganese nodules- which accidentally started everyone else investing in deep sea mining.

The Glomar Explorer was also a breakthrough in computer control, as the ship had to stay on station and move the arm to the sub in the presence of wind, waves, and currents. All with an array of 16-bit microprocessor, 5MHz clock, 32K words of core memory Honeywell computers. Consider that a late model iPhone uses a 64-bit microprocessor, a 3GHz clock, 6GB of RAM and a GPU.

Turtledove takes one major liberty with the otherwise hard science retrospective: the CIA recovering the Soviet sub was in turn a cover story masking the real mission to salvage the alien space ship that apparently collided with the sub!

The dry humor and attention to scientific details makes for an entertaining sci-fi compare-and-contrast between deep sea robotics and computers in the 1970s and the present day. It’s a fun read- not just for roboticists and computer scientists.

For further robotics science reading:

• about ROVs: https://rov.org/history/
• about the automatic station keeping system: The Jennifer Project

For further scifi reading, check out:

• early (1907) and start of underwater robots in scifi at cyberneticzoo.com
• reddit list

View Details

Image/Shutterstock.com

By Angharad Brewer Gillham, Frontiers science writer

‘Social loafing’ is a phenomenon which happens when members of a team start to put less effort in because they know others will cover for them. Scientists investigating whether this happens in teams which combine work by robots and humans found that humans carrying out quality assurance tasks spotted fewer errors when they had been told that robots had already checked a piece, suggesting they relied on the robots and paid less attention to the work.

Now that improvements in technology mean that some robots work alongside humans, there is evidence that those humans have learned to see them as team-mates — and teamwork can have negative as well as positive effects on people’s performance. People sometimes relax, letting their colleagues do the work instead. This is called ‘social loafing’, and it’s common where people know their contribution won’t be noticed or they’ve acclimatized to another team member’s high performance. Scientists at the Technical University of Berlin investigated whether humans social loaf when they work with robots.

“Teamwork is a mixed blessing,” said Dietlind Helene Cymek, first author of the study in Frontiers in Robotics and AI. “Working together can motivate people to perform well but it can also lead to a loss of motivation because the individual contribution is not as visible. We were interested in whether we could also find such motivational effects when the team partner is a robot.”

A helping handThe scientists tested their hypothesis using a simulated industrial defect-inspection task: looking at circuit boards for errors. The scientists provided images of circuit boards to 42 participants. The circuit boards were blurred, and the sharpened images could only be viewed by holding a mouse tool over them. This allowed the scientists to track participants’ inspection of the board.

Half of the participants were told that they were working on circuit boards that had been inspected by a robot called Panda. Although these participants did not work directly with Panda, they had seen the robot and could hear it while they worked. After examining the boards for errors and marking them, all participants were asked to rate their own effort, how responsible for the task they felt, and how they performed.

Looking but not seeingAt first sight, it looked as if the presence of Panda had made no difference — there was no statistically significant difference between the groups in terms of time spent inspecting the circuit boards and the area searched. Participants in both groups rated their feelings of responsibility for the task, effort expended, and performance similarly.

But when the scientists looked more closely at participants’ error rates, they realized that the participants working with Panda were catching fewer defects later in the task, when they’d already seen that Panda had successfully flagged many errors. This could reflect a ‘looking but not seeing’ effect, where people get used to relying on something and engage with it less mentally. Although the participants thought they were paying an equivalent amount of attention, subconsciously they assumed that Panda hadn’t missed any defects.

“It is easy to track where a person is looking, but much harder to tell whether that visual information is being sufficiently processed at a mental level,” said Dr Linda Onnasch, senior author of the study.

The experimental set-up with the human-robot team. Image supplied by the authors.

Safety at risk?The authors warned that this could have safety implications. “In our experiment, the subjects worked on the task for about 90 minutes, and we already found that fewer quality errors were detected when they worked in a team,” said Onnasch. “In longer shifts, when tasks are routine and the working environment offers little performance monitoring and feedback, the loss of motivation tends to be much greater. In manufacturing in general, but especially in safety-related areas where double checking is common, this can have a negative impact on work outcomes.”

The scientists pointed out that their test has some limitations. While participants were told they were in a team with the robot and shown its work, they did not work directly with Panda. Additionally, social loafing is hard to simulate in the laboratory because participants know they are being watched.

“The main limitation is the laboratory setting,” Cymek explained. “To find out how big the problem of loss of motivation is in human-robot interaction, we need to go into the field and test our assumptions in real work environments, with skilled workers who routinely do their work in teams with robots.”

• PAPER – Lean back or lean in? Exploring social loafing in human–robot teams. Dietlind Helene Cymek, Anna Truckenbrodt, and Linda Onnasch. Frontiers in Robotics and AI, 10, 1249252.

View Details

Claire chatted to Ffion Llewellyn from Oshen about sea-faring robots and ocean sensing.

Ffion Llewellyn graduated from Imperial College London in 2022 with a masters in Aeronautical Engineering. Following this, she joined Oshen who are building low cost, autonomous micro-vessels for remote ocean sensing. Ffion has been focused on the integration and testing of sensors onto Oshen’s autonomous micro-vessels, including metocean sensors and hydrophones for the monitoring of marine mammals. Her role also includes the design and manufacture of the micro-vessels, conducting sea trials and analysing the data collected.

View Details

Code to Joy: Why Everyone Should Learn a Little Programming is a new book from Michael Littman, Professor of Computer Science at Brown University and a founding trustee of AIhub. We spoke to Michael about what the book covers, what inspired it, and how we are all familiar with many programming concepts in our daily lives, whether we realize it or not.

Could you start by telling us a bit about the book, and who the intended audience is?The intended audience is not computer scientists, although I have been getting a very warm reception from computer scientists, which I appreciate. The idea behind the book is to try to help people understand that telling machines what to do (which is how I view much of computer science and AI) is something that is really accessible to everyone. It builds on skills and practices that people already have. I think it can be very intimidating for a lot of people, but I don’t think it needs to be. I think that the foundation is there for everybody and it’s just a matter of tapping into that and building on top of it. What I’m hoping, and what I’m seeing happening, is that machine learning and AI is helping to meet people part way. The machines are getting better at listening as we try to get better at telling them what to do.

What made you decide to write the book, what was the inspiration behind it?I’ve taught large introductory computer science classes and I feel like there’s an important message in there about how a deeper knowledge of computing can be very empowering, and I wanted to bring that to a larger audience.

Could you talk a bit about the structure of the book?The meat of the book talks about the fundamental components that make up programs, or, in other words, that make up the way that we tell computers what to do. Each chapter covers a different one of those topics – loops, variables, conditionals, for example. Within each chapter I talk about the ways in which this concept is already familiar to people, the ways that it shows up in regular life. I point to existing pieces of software or websites where you can make use of that one particular concept to tell computers what to do. Each chapter ends with an introduction to some concepts from machine learning that can help create that particular programming construct. For example, in the chapter on conditionals, I talk about the ways that we use the word “if” in regular life all the time. Weddings, for example, are very conditionally structured, with statements like “if anyone has anything to say, speak now or forever hold your peace”. That’s kind of an “if-then” statement. In terms of tools to play with, I talk about interactive fiction. Partway between video games and novels is this notion that you can make a story that adapts itself while it’s being read. What makes that interesting is this notion of conditionals – the reader can make a choice and that will cause a branch. There are really wonderful tools for being able to play with this idea online, so you don’t have to be a full-fledged programmer to make use of conditionals. The machine learning concept introduced there is decision trees, which is an older form of machine learning where you give a system a bunch of examples and then it outputs a little flowchart for decision making.

Do you touch on generative AI in the book?The book was already in production by the time ChatGPT came out, but I was ahead of the curve, and I did have a section specifically about GPT-3 (pre-ChatGPT) which talks about what it is, how machine learning creates it, and how it itself can be helpful in making programs. So, you see it from both directions. You get the notion that this tool actually helps people tell machines what to do, and also the way that humanity created this tool in the first place using machine learning.

Did you learn anything while you were writing the book that was particularly interesting or surprising?Researching the examples for each chapter caused me to dig into a whole bunch of topics. This notion of interactive fiction, and that there’s tools for creating interactive fiction, I found pretty interesting. When researching another chapter, I found an example from a Jewish prayer book that was just so shocking to me. So, Jewish prayer books (and I don’t know if this is true in other belief systems as well, but I’m mostly familiar with Judaism), contain things you’re supposed to read, but they have little conditional markings on them sometimes. For example, one might say “don’t read this if it’s a Saturday”, or “don’t read this if it’s a full moon”, or “don’t read if it’s a full moon on a Saturday”. I found one passage that actually had 14 different conditions that you had to check to decide whether or not it was appropriate to read this particular passage. That was surprising to me – I had no idea that people were expected to do so much complex computation during a worship activity.

Why is it important that everybody learns a little programming?It’s really important to keep in mind the idea that at the end of the day what AI is doing is making it easier for us to tell machines what to do, and we should share that increased capability with a broad population. It shouldn’t just be the machine learning engineers who get to tell computers what to do more easily. We should find ways of making this easier for everybody.

Because computers are here to help, but it’s a two-way street. We need to be willing to learn to express what we want in a way that can be carried out accurately and automatically. If we don’t make that effort, then other parties, companies often, will step in and do it for us. At that point, the machines are working to serve some else’s interest instead of our own. I think it’s become absolutely essential that we restore a healthy relationship with these machines before we lose any more of our autonomy.

Any final thoughts or takeaways that we should bear in mind?I think there’s a message here for computer science researchers, as well. When we tell other people what to do, we tend to combine a description or a rule, something that’s sort of program-like, with examples, something that’s more data-like. We just intermingle them when we talk to each other. At one point when I was writing the book, I had a dishwasher that was acting up and I wanted to understand why. I read through its manual, and I was struck by how often it was the case that in telling people what to do with the dishwasher, the authors would consistently mix together a high-level description of what they are telling you to do with some particular, vivid examples: a rule for what to load into the top rack, and a list of items that fit that rule. That seems to be the way that people want to both convey and receive information. What’s crazy to me is that we don’t program computers that way. We either use something that’s strictly programming, all rules, no examples, or we use machine learning, where it’s all examples, no rules. I think the reason that people communicate this way with each other is because those two different mechanisms have complementary strengths and weaknesses and when you combine the two together, you maximize the chance of being accurately understood. And that’s the goal when we’re telling machines what to do. I want the AI community to be thinking about how we can combine what we’ve learned about machine learning with something more programming-like to make a much more powerful way of telling machines what to do. I don’t think this is a solved problem yet, and that’s something that I really hope that people in the community think about.

Code to Joy: Why Everyone Should Learn a Little Programming is available to buy now.

| | Michael L. Littman is a University Professor of Computer Science at Brown University, studying machine learning and decision making under uncertainty. He has earned multiple university-level awards for teaching and his research on reinforcement learning, probabilistic planning, and automated crossword-puzzle solving has been recognized with three best-paper awards and three influential paper awards. Littman is co-director of Brown’s Humanity Centered Robotics Initiative and a Fellow of the Association for the Advancement of Artificial Intelligence and the Association for Computing Machinery. He is also a Fellow of the American Association for the Advancement of Science Leshner Leadership Institute for Public Engagement with Science, focusing on Artificial Intelligence. He is currently serving as Division Director for Information and Intelligent Systems at the National Science Foundation. |

View Details

Source: OpenAI’s DALL·E 2 with prompt “a hyperrealistic picture of a robot reading the news on a laptop at a coffee shop”

Welcome to the 6th edition of Robo-Insight, a robotics news update! In this post, we are excited to share a range of new advancements in the field and highlight robots’ progress in areas like medical assistance, prosthetics, robot flexibility, joint movement, work performance, AI design, and household cleanliness.

Robots that can aid nursesIn the medical world, researchers from Germany have developed a robotic system designed to help nurses relieve the physical strain associated with patient care. Nurses often face high physical demands when attending to bedridden patients, especially during tasks like repositioning them. Their work explores how robotic technology can assist in such tasks by remotely anchoring patients in a lateral position. The results indicate that the system improved the working posture of nurses by an average of 11.93% and was rated as user-friendly. The research highlights the potential for robotics to support caregivers in healthcare settings, improving both nurse working conditions and patient care.

Arrangement of patient room used in the study. Source.

Robots enhancing bionic hand controlKeeping our focus healthcare-related, recently researchers from numerous European institutions have achieved a significant breakthrough in robot prosthetic technology, as they successfully implanted a neuromusculoskeletal prosthesis, a bionic hand connected directly to the user’s nervous and skeletal systems, in a person with a below-elbow amputation. This achievement involved surgical procedures to place titanium implants in the radius and ulna bones and transfer severed nerves to free muscle grafts. These neural interfaces provided a direct connection between the prosthesis and the user’s body, allowing for improved prosthetic function and increased quality of life. Their work demonstrates the potential for highly integrated prosthetic devices to enhance the lives of amputees through reliable neural control and comfortable, everyday use.

Schematic and X-ray of a fully integrated human-machine interface in a patient. Source.

Reinforcement learning in soft roboticsTurning our focus to soft robotics, researchers from the Center for Research and Advanced Studies of the National Polytechnic Institut of Mexico and the Universidad Autónoma de Coahuila have proposed an approach to use reinforcement learning (RL) for motor control of a pneumatic-driven soft robot modeled after continuum media with varying density. This method involves a continuous-time Actor-Critic scheme designed for tracking tasks in a 3D soft robot subject to Lipschitz disturbances. Their study introduces a reward-based temporal difference mechanism and a discontinuous adaptive approach for neural weights in the Critic component of the system. The overall aim is to enable RL to control the complex, uncertain, and deformable nature of soft robots while ensuring stability in real-time control, a crucial requirement for physical systems. This research focuses on the application of RL in managing the unique challenges posed by soft robots.

Distinct distortions of a cylindrical-shaped flexible robot. Source.

A teen-sized humanoid robotMoving onto human-robot interactions, researchers from the University of Texas at Austin’s Human-Centered Robotics Laboratory have introduced a teen-sized humanoid robot named DRACO 3, designed in collaboration with Apptronik. This robot, tailored for practical use in human environments, features proximal actuation and employs rolling contact mechanisms on its lower body, allowing for extensive vertical poses. A whole-body controller (WBC) has been developed to manage DRACO 3’s complex transmissions. This research offers insights into the development and control of humanoids with rolling contact joints, focusing on practicality and performance.

Diagram illustrating the rolling contact joint at the knee. Initial configuration (left) and post-angular displacement (right). Source.

Robots’ impacts on performanceShifting our focus to psychology, recently researchers from Technische Universität Berlin have investigated the phenomenon of social loafing in human-robot teams. Social loafing refers to reduced individual effort in a team setting compared to working alone. The study involved participants inspecting circuit boards for defects, with one group working alone and the other with a robot partner. Despite a reliable robot that marked defects on boards, participants working with the robot identified fewer defects compared to those working alone, suggesting a potential occurrence of social loafing in human-robot teams. This research sheds light on the challenges associated with human-robot collaboration and its impact on individual effort and performance.

Results of solo work vs. robot work. Source.

A robot designed by AIChanging our focus to robot design, researchers from Northwestern University have developed an AI system that designs robots from scratch, enabling it to create a walking robot in seconds, a task that took nature billions of years to evolve. This AI system runs on a lightweight personal computer, without relying on energy-hungry supercomputers or large datasets, offering the potential to design robots with unique forms rapidly. The system works by iterating on a design, assessing its flaws, and refining the structure in a matter of seconds. It paves the way for a new era of AI-designed tools capable of acting directly on the world for various applications.

3D printer designing robot. Source.

A customizable robot for household organizationFinally, in the field of home robotics, researchers from Stanford, Princeton, Columbia University, and Google, have developed TidyBot, a one-armed robot designed to clean spaces according to personal preferences. TidyBot uses a large language model trained on internet data to identify various objects and understand where to put them, making it highly customizable to different preferences. In real-world tests, the robot can correctly put away approximately 85% of objects, significantly improving household organization. While TidyBot still has room for improvement, researchers believe it holds great promise for making robots more versatile and useful in homes and other environments.

Tidybot in training. Source.

The ongoing development in a multitude of sectors highlights the flexibility and steadily advancing character of robotics technology, uncovering fresh possibilities for its incorporation into a wide range of industries. The progressive expansion in the realm of robotics mirrors unwavering commitment and offers a glimpse into the potential consequences of these advancements for the times ahead.

Sources:

1. Hinrichs, P., Seibert, K., Arizpe Gómez, P., Pfingsthorn, M., & Hein, A. (2023). A Robotic System to Anchor a Patient in a Lateral Position and Reduce Nurses’ Physical Strain. Robotics, 12(5)
2. Ortiz-Catalán, M., Zbinden, J., Millenaar, J., D’Accolti, D., Controzzi, M., Clemente, F., Cappello, L., Earley, E. J., Enzo Mastinu, Justyna Kolankowska, Munoz-Novoa, M., Stewe Jönsson, Njel, C., Paolo Sassu, & Rickard Brånemark. (2023). A highly integrated bionic hand with neural control and feedback for use in daily life. Science Robotics
3. Pantoja-Garcia, L., Parra-Vega, V., Garcia-Rodriguez, R., & Vázquez-García, C. E. (2023). A Novel Actor—Critic Motor Reinforcement Learning for Continuum Soft Robots. Robotics, 12(5)
4. Bang, S. H., Gonzalez, C., Ahn, J., Paine, N., & Sentis, L. (2023, September 26). Control and evaluation of a humanoid robot with rolling contact joints on its lower body. Frontiers.
5. Cymek, D. H., Truckenbrodt, A., & Onnasch, L. (2023, August 31). Lean back or lean in? exploring social loafing in human–robot teams. Frontiers.
6. Instant evolution: AI designs new robot from scratch in seconds. (n.d.). News.northwestern.edu.
7. University, S. (2023, October 3). Robot provides personalized room cleanup. Stanford News.

View Details

By Andre He, Vivek Myers

A longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans. Natural language has the potential to be an easy-to-use interface for humans to specify arbitrary tasks, but it is difficult to train robots to follow language instructions. Approaches like language-conditioned behavioral cloning (LCBC) train policies to directly imitate expert actions conditioned on language, but require humans to annotate all training trajectories and generalize poorly across scenes and behaviors. Meanwhile, recent goal-conditioned approaches perform much better at general manipulation tasks, but do not enable easy task specification for human operators. How can we reconcile the ease of specifying tasks through LCBC-like approaches with the performance improvements of goal-conditioned learning?

Conceptually, an instruction-following robot requires two capabilities. It needs to ground the language instruction in the physical environment, and then be able to carry out a sequence of actions to complete the intended task. These capabilities do not need to be learned end-to-end from human-annotated trajectories alone, but can instead be learned separately from the appropriate data sources. Vision-language data from non-robot sources can help learn language grounding with generalization to diverse instructions and visual scenes. Meanwhile, unlabeled robot trajectories can be used to train a robot to reach specific goal states, even when they are not associated with language instructions.

Conditioning on visual goals (i.e. goal images) provides complementary benefits for policy learning. As a form of task specification, goals are desirable for scaling because they can be freely generated hindsight relabeling (any state reached along a trajectory can be a goal). This allows policies to be trained via goal-conditioned behavioral cloning (GCBC) on large amounts of unannotated and unstructured trajectory data, including data collected autonomously by the robot itself. Goals are also easier to ground since, as images, they can be directly compared pixel-by-pixel with other states.

However, goals are less intuitive for human users than natural language. In most cases, it is easier for a user to describe the task they want performed than it is to provide a goal image, which would likely require performing the task anyways to generate the image. By exposing a language interface for goal-conditioned policies, we can combine the strengths of both goal- and language- task specification to enable generalist robots that can be easily commanded. Our method, discussed below, exposes such an interface to generalize to diverse instructions and scenes using vision-language data, and improve its physical skills by digesting large unstructured robot datasets.

Goal representations for instruction followingThe GRIF model consists of a language encoder, a goal encoder, and a policy network. The encoders respectively map language instructions and goal images into a shared task representation space, which conditions the policy network when predicting actions. The model can effectively be conditioned on either language instructions or goal images to predict actions, but we are primarily using goal-conditioned training as a way to improve the language-conditioned use case.

Our approach, Goal Representations for Instruction Following (GRIF), jointly trains a language- and a goal- conditioned policy with aligned task representations. Our key insight is that these representations, aligned across language and goal modalities, enable us to effectively combine the benefits of goal-conditioned learning with a language-conditioned policy. The learned policies are then able to generalize across language and scenes after training on mostly unlabeled demonstration data.

We trained GRIF on a version of the Bridge-v2 dataset containing 7k labeled demonstration trajectories and 47k unlabeled ones within a kitchen manipulation setting. Since all the trajectories in this dataset had to be manually annotated by humans, being able to directly use the 47k trajectories without annotation significantly improves efficiency.

To learn from both types of data, GRIF is trained jointly with language-conditioned behavioral cloning (LCBC) and goal-conditioned behavioral cloning (GCBC). The labeled dataset contains both language and goal task specifications, so we use it to supervise both the language- and goal-conditioned predictions (i.e. LCBC and GCBC). The unlabeled dataset contains only goals and is used for GCBC. The difference between LCBC and GCBC is just a matter of selecting the task representation from the corresponding encoder, which is passed into a shared policy network to predict actions.

By sharing the policy network, we can expect some improvement from using the unlabeled dataset for goal-conditioned training. However,GRIF enables much stronger transfer between the two modalities by recognizing that some language instructions and goal images specify the same behavior. In particular, we exploit this structure by requiring that language- and goal- representations be similar for the same semantic task. Assuming this structure holds, unlabeled data can also benefit the language-conditioned policy since the goal representation approximates that of the missing instruction.

Alignment through contrastive learningWe explicitly align representations between goal-conditioned and language-conditioned tasks on the labeled dataset through contrastive learning.

Since language often describes relative change, we choose to align representations of state-goal pairs with the language instruction (as opposed to just goal with language). Empirically, this also makes the representations easier to learn since they can omit most information in the images and focus on the change from state to goal.

We learn this alignment structure through an infoNCE objective on instructions and images from the labeled dataset. We train dual image and text encoders by doing contrastive learning on matching pairs of language and goal representations. The objective encourages high similarity between representations of the same task and low similarity for others, where the negative examples are sampled from other trajectories.

When using naive negative sampling (uniform from the rest of the dataset), the learned representations often ignored the actual task and simply aligned instructions and goals that referred to the same scenes. To use the policy in the real world, it is not very useful to associate language with a scene; rather we need it to disambiguate between different tasks in the same scene. Thus, we use a hard negative sampling strategy, where up to half the negatives are sampled from different trajectories in the same scene.

Naturally, this contrastive learning setup teases at pre-trained vision-language models like CLIP. They demonstrate effective zero-shot and few-shot generalization capability for vision-language tasks, and offer a way to incorporate knowledge from internet-scale pre-training. However, most vision-language models are designed for aligning a single static image with its caption without the ability to understand changes in the environment, and they perform poorly when having to pay attention to a single object in cluttered scenes.

To address these issues, we devise a mechanism to accommodate and fine-tune CLIP for aligning task representations. We modify the CLIP architecture so that it can operate on a pair of images combined with early fusion (stacked channel-wise). This turns out to be a capable initialization for encoding pairs of state and goal images, and one which is particularly good at preserving the pre-training benefits from CLIP.

Robot policy resultsFor our main result, we evaluate the GRIF policy in the real world on 15 tasks across 3 scenes. The instructions are chosen to be a mix of ones that are well-represented in the training data and novel ones that require some degree of compositional generalization. One of the scenes also features an unseen combination of objects.

We compare GRIF against plain LCBC and stronger baselines inspired by prior work like LangLfP and BC-Z. LLfP corresponds to jointly training with LCBC and GCBC. BC-Z is an adaptation of the namesake method to our setting, where we train on LCBC, GCBC, and a simple alignment term. It optimizes the cosine distance loss between the task representations and does not use image-language pre-training.

The policies were susceptible to two main failure modes. They can fail to understand the language instruction, which results in them attempting another task or performing no useful actions at all. When language grounding is not robust, policies might even start an unintended task after having done the right task, since the original instruction is out of context.

Examples of grounding failures

“put the mushroom in the metal pot”

“put the spoon on the towel”

“put the yellow bell pepper on the cloth”

“put the yellow bell pepper on the cloth”

The other failure mode is failing to manipulate objects. This can be due to missing a grasp, moving imprecisely, or releasing objects at the incorrect time. We note that these are not inherent shortcomings of the robot setup, as a GCBC policy trained on the entire dataset can consistently succeed in manipulation. Rather, this failure mode generally indicates an ineffectiveness in leveraging goal-conditioned data.

Examples of manipulation failures

“move the bell pepper to the left of the table”

“put the bell pepper in the pan”

“move the towel next to the microwave”

Comparing the baselines, they each suffered from these two failure modes to different extents. LCBC relies solely on the small labeled trajectory dataset, and its poor manipulation capability prevents it from completing any tasks. LLfP jointly trains the policy on labeled and unlabeled data and shows significantly improved manipulation capability from LCBC. It achieves reasonable success rates for common instructions, but fails to ground more complex instructions. BC-Z’s alignment strategy also improves manipulation capability, likely because alignment improves the transfer between modalities. However, without external vision-language data sources, it still struggles to generalize to new instructions.

GRIF shows the best generalization while also having strong manipulation capabilities. It is able to ground the language instructions and carry out the task even when many distinct tasks are possible in the scene. We show some rollouts and the corresponding instructions below.

Policy Rollouts from GRIF

“move the pan to the front”

“put the bell pepper in the pan”

“put the knife on the purple cloth”

“put the spoon on the towel”

ConclusionGRIF enables a robot to utilize large amounts of unlabeled trajectory data to learn goal-conditioned policies, while providing a “language interface” to these policies via aligned language-goal task representations. In contrast to prior language-image alignment methods, our representations align changes in state to language, which we show leads to significant improvements over standard CLIP-style image-language alignment objectives. Our experiments demonstrate that our approach can effectively leverage unlabeled robotic trajectories, with large improvements in performance over baselines and methods that only use the language-annotated data

Our method has a number of limitations that could be addressed in future work. GRIF is not well-suited for tasks where instructions say more about how to do the task than what to do (e.g., “pour the water slowly”)—such qualitative instructions might require other types of alignment losses that consider the intermediate steps of task execution. GRIF also assumes that all language grounding comes from the portion of our dataset that is fully annotated or a pre-trained VLM. An exciting direction for future work would be to extend our alignment loss to utilize human video data to learn rich semantics from Internet-scale data. Such an approach could then use this data to improve grounding on language outside the robot dataset and enable broadly generalizable robot policies that can follow user instructions.

This post is based on the following paper:

• Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control
  Vivek Myers, Andre He, Kuan Fang, Homer Walke, Philippe Hansen-Estruch, Ching-An Cheng, Mihai Jalobeanu, Andrey Kolobov, Anca Dragan, and Sergey Levine

View Details

Claire chatted to Lorenzo Jamone from Queen Mary University of London all about robotic hands, dexterity, and the sense of touch.

Lorenzo Jamone is a Senior Lecturer in Robotics at Queen Mary University of London, where he is the founder and director of the CRISP group: Cognitive Robotics and Intelligent Systems for the People. He received a PhD degree in humanoid technologies at the Italian Institute of Technology. He was previously an Associate Researcher at Waseda University in Japan, and at the Instituto Superior Técnico in Portugal. His current research interests include cognitive robotics, robotic manipulation, force and tactile sensing, robot learning.

View Details

MIT researchers are using generative AI models to help robots more efficiently solve complex object manipulation problems, such as packing a box with different objects. Image: courtesy of the researchers.

By Adam Zewe | MIT News

Anyone who has ever tried to pack a family-sized amount of luggage into a sedan-sized trunk knows this is a hard problem. Robots struggle with dense packing tasks, too.

For the robot, solving the packing problem involves satisfying many constraints, such as stacking luggage so suitcases don’t topple out of the trunk, heavy objects aren’t placed on top of lighter ones, and collisions between the robotic arm and the car’s bumper are avoided.

Some traditional methods tackle this problem sequentially, guessing a partial solution that meets one constraint at a time and then checking to see if any other constraints were violated. With a long sequence of actions to take, and a pile of luggage to pack, this process can be impractically time consuming.

MIT researchers used a form of generative AI, called a diffusion model, to solve this problem more efficiently. Their method uses a collection of machine-learning models, each of which is trained to represent one specific type of constraint. These models are combined to generate global solutions to the packing problem, taking into account all constraints at once.

Their method was able to generate effective solutions faster than other techniques, and it produced a greater number of successful solutions in the same amount of time. Importantly, their technique was also able to solve problems with novel combinations of constraints and larger numbers of objects, that the models did not see during training.

Due to this generalizability, their technique can be used to teach robots how to understand and meet the overall constraints of packing problems, such as the importance of avoiding collisions or a desire for one object to be next to another object. Robots trained in this way could be applied to a wide array of complex tasks in diverse environments, from order fulfillment in a warehouse to organizing a bookshelf in someone’s home.

“My vision is to push robots to do more complicated tasks that have many geometric constraints and more continuous decisions that need to be made — these are the kinds of problems service robots face in our unstructured and diverse human environments. With the powerful tool of compositional diffusion models, we can now solve these more complex problems and get great generalization results,” says Zhutian Yang, an electrical engineering and computer science graduate student and lead author of a paper on this new machine-learning technique.

Her co-authors include MIT graduate students Jiayuan Mao and Yilun Du; Jiajun Wu, an assistant professor of computer science at Stanford University; Joshua B. Tenenbaum, a professor in MIT’s Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Tomás Lozano-Pérez, an MIT professor of computer science and engineering and a member of CSAIL; and senior author Leslie Kaelbling, the Panasonic Professor of Computer Science and Engineering at MIT and a member of CSAIL. The research will be presented at the Conference on Robot Learning.

Constraint complicationsContinuous constraint satisfaction problems are particularly challenging for robots. These problems appear in multistep robot manipulation tasks, like packing items into a box or setting a dinner table. They often involve achieving a number of constraints, including geometric constraints, such as avoiding collisions between the robot arm and the environment; physical constraints, such as stacking objects so they are stable; and qualitative constraints, such as placing a spoon to the right of a knife.

There may be many constraints, and they vary across problems and environments depending on the geometry of objects and human-specified requirements.

To solve these problems efficiently, the MIT researchers developed a machine-learning technique called Diffusion-CCSP. Diffusion models learn to generate new data samples that resemble samples in a training dataset by iteratively refining their output.

To do this, diffusion models learn a procedure for making small improvements to a potential solution. Then, to solve a problem, they start with a random, very bad solution and then gradually improve it.

Using generative AI models, MIT researchers created a technique that could enable robots to efficiently solve continuous constraint satisfaction problems, such as packing objects into a box while avoiding collisions, as shown in this simulation. Image: Courtesy of the researchers.

For example, imagine randomly placing plates and utensils on a simulated table, allowing them to physically overlap. The collision-free constraints between objects will result in them nudging each other away, while qualitative constraints will drag the plate to the center, align the salad fork and dinner fork, etc.

Diffusion models are well-suited for this kind of continuous constraint-satisfaction problem because the influences from multiple models on the pose of one object can be composed to encourage the satisfaction of all constraints, Yang explains. By starting from a random initial guess each time, the models can obtain a diverse set of good solutions.

Working togetherFor Diffusion-CCSP, the researchers wanted to capture the interconnectedness of the constraints. In packing for instance, one constraint might require a certain object to be next to another object, while a second constraint might specify where one of those objects must be located.

Diffusion-CCSP learns a family of diffusion models, with one for each type of constraint. The models are trained together, so they share some knowledge, like the geometry of the objects to be packed.

The models then work together to find solutions, in this case locations for the objects to be placed, that jointly satisfy the constraints.

“We don’t always get to a solution at the first guess. But when you keep refining the solution and some violation happens, it should lead you to a better solution. You get guidance from getting something wrong,” she says.

Training individual models for each constraint type and then combining them to make predictions greatly reduces the amount of training data required, compared to other approaches.

However, training these models still requires a large amount of data that demonstrate solved problems. Humans would need to solve each problem with traditional slow methods, making the cost to generate such data prohibitive, Yang says.

Instead, the researchers reversed the process by coming up with solutions first. They used fast algorithms to generate segmented boxes and fit a diverse set of 3D objects into each segment, ensuring tight packing, stable poses, and collision-free solutions.

“With this process, data generation is almost instantaneous in simulation. We can generate tens of thousands of environments where we know the problems are solvable,” she says.

Trained using these data, the diffusion models work together to determine locations objects should be placed by the robotic gripper that achieve the packing task while meeting all of the constraints.

They conducted feasibility studies, and then demonstrated Diffusion-CCSP with a real robot solving a number of difficult problems, including fitting 2D triangles into a box, packing 2D shapes with spatial relationship constraints, stacking 3D objects with stability constraints, and packing 3D objects with a robotic arm.

This figure shows examples of 2D triangle packing. These are collision-free configurations. Image: courtesy of the researchers.

This figure shows 3D object stacking with stability constraints. Researchers say at least one object is supported by multiple objects. Image: courtesy of the researchers.

Their method outperformed other techniques in many experiments, generating a greater number of effective solutions that were both stable and collision-free.

In the future, Yang and her collaborators want to test Diffusion-CCSP in more complicated situations, such as with robots that can move around a room. They also want to enable Diffusion-CCSP to tackle problems in different domains without the need to be retrained on new data.

“Diffusion-CCSP is a machine-learning solution that builds on existing powerful generative models,” says Danfei Xu, an assistant professor in the School of Interactive Computing at the Georgia Institute of Technology and a Research Scientist at NVIDIA AI, who was not involved with this work. “It can quickly generate solutions that simultaneously satisfy multiple constraints by composing known individual constraint models. Although it’s still in the early phases of development, the ongoing advancements in this approach hold the promise of enabling more efficient, safe, and reliable autonomous systems in various applications.”

This research was funded, in part, by the National Science Foundation, the Air Force Office of Scientific Research, the Office of Naval Research, the MIT-IBM Watson AI Lab, the MIT Quest for Intelligence, the Center for Brains, Minds, and Machines, Boston Dynamics Artificial Intelligence Institute, the Stanford Institute for Human-Centered Artificial Intelligence, Analog Devices, JPMorgan Chase and Co., and Salesforce.

• PAPER – Compositional Diffusion-Based Continuous Constraint Solvers. Zhutian Yang, Jiayuan Mao, Yilun Du, Jiajun Wu,
  Joshua B. Tenenbaum, Tomás Lozano-Pérez, and Leslie Pack Kaelbling. arXive.

View Details

A RoboCupJunior soccer match in action.

In July this year, 2500 participants congregated in Bordeaux for RoboCup2023. The competition comprises a number of leagues, and among them is RoboCupJunior, which is designed to introduce RoboCup to school children, with the focus being on education. There are three sub-leagues: Soccer, Rescue and OnStage.

Marek Šuppa serves on the Executive Committee for RoboCupJunior, and he told us about the competition this year and the latest developments in the Soccer league.

What is your role in RoboCupJunior and how long have you been involved with this league?I started with RoboCupJunior quite a while ago: my first international competition was in 2009 in Graz, where I was lucky enough to compete in Soccer for the first time. Our team didn’t do all that well in that event but RoboCup made a deep impression and so I stayed around: first as a competitor and later to help organise the RoboCupJunior Soccer league. Right now I am serving as part of the RoboCupJunior Execs who are responsible for the organisation of RoboCupJunior as a whole.

How was the event this year? What were some of the highlights? I guess this year’s theme or slogan, if we were to give it one, would be “back to normal”, or something like that. Although RoboCup 2022 already took place in-person in Thailand last year after two years of a pandemic pause, it was in a rather limited capacity, as COVID-19 still affected quite a few regions. It was great to see that the RoboCup community was able to persevere and even thrive throughout the pandemic, and that RoboCup 2023 was once again an event where thousands of robots and roboticists meet.

It would also be difficult to do this question justice without thanking the local French organisers. They were actually ready to organise the event in 2020 but it got cancelled due to COVID-19. But they did not give up on the idea and managed to put together an awesome event this year, for which we are very thankful.

Examples of the robots used by the RoboCupJunior Soccer teams.

Turning to RoboCupJunior Soccer specifically, could you talk about the mission of the league and how you, as organisers, go about realising that mission?The mission of RoboCupJunior consists of two competing objectives: on one hand, it needs to be a challenge that’s approachable, interesting and relevant for (mostly) high school students and at the same time it needs to be closely related to the RoboCup “Major” challenges, which are tackled by university students and their mentors. We are hence continuously trying to both make it more compelling and captivating for the students and at the same time ensure it is technical enough to help them grow towards the RoboCup “Major” challenges.

One of the ways we do that is by introducing what we call “SuperTeam” challenges, in which teams from respective countries form a so-called “SuperTeam” and compete against another “SuperTeam” as if these were distinct teams. In RoboCupJunior Soccer the “SuperTeams” are composed of four to five teams and they compete on a field that is six times larger than the “standard” fields that are used for the individual games. While in the individual matches each team can play with two robots at most (resulting in a 2v2 game) in a SuperTeam match each SuperTeam fields five robots, meaning there are 10 robots that play on the SuperTeam field during a SuperTeam match. The setup is very similar to the Division B of the Small Size League of RoboCup “Major”.

The SuperTeam games have existed in RoboCupJunior Soccer since 2013, so for quite a while, and the feedback we received on them was overwhelmingly positive: it was a lot of fun for both the participants as well as the spectators. But compared to the Small Size League games there were still two noticeable differences: the robots did not have a way of communicating with one another and additionally, the referees did not have a way of communicating with the robots. The result was that not only was there little coordination among robots of the same SuperTeam, whenever the game needed to be stopped, the referees had to physically run after the robots on the field to catch them and do a kickoff after a goal was scored. Although hilarious, it’s far from how we would imagine the SuperTeam games to look.

The RoboCupJunior Soccer Standard Communication Modules aim to do both. The module itself is a small device that is attached to each robot on the SuperTeam field. These devices are all connected via Bluetooth to a single smartphone, through which the referee can send commands to all robots on the field. The devices themselves also support direct message exchange between robots on a single SuperTeam, meaning the teams do not have to invest into figuring out how to communicate with the other robots but can make use of a common platform. The devices, as well as their firmware, are open source, meaning not only that everyone can build their own Standard Communication Module if they’d like but also that the community can participate in its development, which makes it an interesting addition to RoboCupJunior Soccer.

RoboCupJunior Soccer teams getting ready for the competition.

How did this new module work out in the competition? Did you see an improvement in experience for the teams and organisers?In this first big public test we focused on exploring how (and whether) these modules can improve the gameplay – especially the “chasing robots at kickoff”. Although we’ve done “lab experiments” in the past and had some empirical evidence that it should work rather well, this was the first time we tried it in a real competition.

All in all, I would say that it was a very positive experiment. The modules themselves did work quite well and for some of us, who happened to have experience with “robot chasing” mentioned above, it was sort of a magical feeling to see the robots stop right on the main referee’s whistle.

We also found out potential areas for improvement in the future. The modules themselves do not have a power source of their own and were powered by the robots themselves. We didn’t think this would be a problem but in the “real world” test it transpired that the voltage levels the robots are capable of providing fluctuates significantly – for instance when the robot decides to aggressively accelerate – which in turn means some of the modules disconnect when the voltage is lowered significantly. However, it ended up being a nice lesson for everyone involved, one that we can certainly learn from when we design the next iterations.

The livestream from Day 4 of RoboCupJunior Soccer 2023. This stream includes the SuperTeam finals and the technical challenges. You can also view the livestream of the semifinals and finals from day three here.

Could you tell us about the emergence of deep-learning models in the RoboCupJunior leagues?This is something we started to observe in recent years which surprised us organisers, to some extent. In our day-to-day jobs (that is, when we are not organising RoboCup), many of us, the organisers, work in areas related to robotics, computer science, and engineering in general – with some of us also doing research in artificial intelligence and machine learning. And while we always thought that it would be great to see more of the cutting-edge research being applied at RoboCupJunior, we always dismissed it as something too advanced and/or difficult to set up for the high school students that comprise the majority of RoboCupJunior students.

Well, to our great surprise, some of the more advanced teams have started to utilise methods and technologies that are very close to the current state-of-the-art in various areas, particularly computer vision and deep learning. A good example would be object detectors (usually based on the YOLO architecture), which are now used across all three Junior leagues: in OnStage to detect various props, robots and humans who perform on the stage together, in Rescue to detect the victims the robots are rescuing and in Soccer to detect the ball, the goals, and the opponents. And while the participants generally used an off-the-shelf implementations, they still needed to do all the steps necessary for a successful deployment of this technology: gather a dataset, finetune the deep-learning model and deploy it on their robots – all of which is far from trivial and is very close to how these technologies get used in both research and industry.

Although we have seen only the more advanced teams use deep-learning models at RoboCupJunior, we expect that in the future we will see it become much more prevalent, especially as the technology and the tooling around it becomes more mature and robust. It does show, however, that despite their age, the RoboCupJunior students are very close to cutting-edge research and state-of-the-art technologies.

Action from RoboCupJunior Soccer 2023.

How can people get involved in RCJ (either as a participant or an organiser?)A very good question!

The best place to start would be the RoboCupJunior website where one can find many interesting details about RoboCupJunior, the respective leagues (such as Soccer, Rescue and OnStage), and the relevant regional representatives who organise regional events. Getting in touch with a regional representative is by far the easiest way of getting started with RoboCup Junior.

Additionally, I can certainly recommend the RoboCupJunior forum, where many RoboCupJunior participants, past and present, as well as the organisers, discuss many related topics in the open. The community is very beginner friendly, so if RoboCupJunior sounds interesting, do not hesitate to stop by and say hi!

About Marek Šuppa

| | Marek stumbled upon AI as a teenager when building soccer-playing robots and quickly realised he is not smart enough to do all the programming by himself. Since then, he’s been figuring out ways to make machines learn by themselves, particularly from text and images. He currently serves as the Principal Data Scientist at Slido (part of Cisco), improving the way meetings are run around the world. Staying true to his roots, he tries to provide others with a chance to have a similar experience by organising the RoboCupJunior competition as part of the Executive Committee. |

View Details

The world’s fourth industrial revolution is ushering in big shifts in the workplace. © demaerre, iStock.com

Professor Steven Dhondt has a reassurance of sorts for people in the EU worried about losing their jobs to automation: relax.

Dhondt, an expert in work and organisational change at the Catholic University Leuven in Belgium, has studied the impact of technology on jobs for the past four decades. Fresh from leading an EU research project on the issue, he stresses opportunities rather than threats.

Right vision‘We need to develop new business practices and welfare support but, with the right vision, we shouldn’t see technology as a threat,’ Dhondt said. ‘Rather, we should use it to shape the future and create new jobs.’

The rapid and accelerating advance in digital technologies across the board is regarded as the world’s fourth industrial revolution, ushering in fundamental shifts in how people live and work.

If the first industrial revolution was powered by steam, the second by electricity and the third by electronics, the latest will be remembered for automation, robotics and artificial intelligence, or AI. It’s known as “Industry 4.0”.

‘Whether it was the Luddite movement in the 1800s through the introduction of automatic spinning machines in the wool industry or concerns about AI today, questions about technology’s impact on jobs really reflect wider ones about employment practices and the labour market,’ said Dhondt.

He is also a senior scientist at a Netherlands-based independent research organisation called TNO.

The EU project that Dhondt led explored how businesses and welfare systems could better adapt to support workers in the face of technological changes. The initiative, called Beyond4.0, began in January 2019 and wrapped up in June 2023.

While the emergence of self-driving cars and AI-assisted robots holds big potential for economic growth and social progress, they also sound alarm bells.

More than 70% of EU citizens fear that new technologies will “steal” people’s jobs, according to a 2019 analysis by the European Centre for the Development of Vocational Training.

Local successesThe Beyond4.0 researchers studied businesses across Europe that have taken proactive and practical steps to empower employees.

“We shouldn’t see technology as a threat – rather we should use it to shape the future and create new jobs.”

– Professor Steven Dhondt, BEYOND4.0

One example is a family-run Dutch glass company called Metaglas, which decided that staying competitive in the face of technological changes required investing more in its own workforce.

Metaglas offered workers greater openness with management and a louder voice on the company’s direction and product development.

The move, which the company named “MetaWay”, has helped it retain workers while turning a profit that is being reinvested in the workforce, according to Dhondt.

He said the example shows the importance in the business world of managers’ approach to the whole issue.

‘The technology can be an enabler, not a threat, but the decision about that lies with management in organisations,’ Dhondt said. ‘If management uses technology to downgrade the quality of jobs, then jobs are at risk. If management uses technology to enhance jobs, then you can see workers and organisations learn and improve.’

The Metaglas case has fed into a “knowledge bank” meant to inform business practices more broadly.

Dhondt also highlighted the importance of regions in Europe where businesses and job trainers join forces to support people.

BEYOND4.0 studied the case of the Finnish city of Oulu – once a leading outpost of mobile-phone giant Nokia. In the 2010s, the demise of Nokia’s handset business threatened Oulu with a “brain drain” as the company’s engineers were laid-off.

But collaboration among Nokia, local universities and policymakers helped grow new businesses including digital spin-offs and kept hundreds of engineers in the central Finnish region, once a trading centre for wood tar, timber and salmon.

Some Nokia engineers went to the local hospital to work on electronic healthcare services – “e-health” – while others moved to papermaker Stora Enso, according to Dhondt.

Nowadays there are more high-tech jobs in Oulu than during Nokia’s heyday. The BEYOND4.0 team held the area up as a successful “entrepreneurial ecosystem” that could help inform policies and practices elsewhere in Europe.

Income supportIn cases where people were out of work, the project also looked to new forms of welfare support.

Dhondt’s Finnish colleagues examined the impact of a two-year trial in Finland of a “universal basic income” – or UBI – and used this to assess the feasibility of a different model called “participation income.”

In the UBI experiment, participants each received a monthly €560 sum, which was paid unconditionally. Although UBI is often touted as an answer to automation, BEYOND4.0’s evaluation of the Finnish trial was that it could weaken the principle of solidarity in society.

The project’s participation income approach requires recipients of financial support to undertake an activity deemed useful to society. This might include, for example, care for the elderly or for children.

While detailed aspects are still being worked out, the BEYOND4.0 team discussed participation income with the government of Finland and the Finnish parliament has put the idea on the agenda for debate.

Dhondt hopes the project’s findings, including on welfare support, will help other organisations better navigate the changing tech landscape.

Employment matchmakersAnother researcher keen to help people adapt to technological changes is Dr Aisling Tuite, a labour-market expert at the South East Technical University in Ireland.

“We wanted to develop a product that could be as useful for people looking for work as for those supporting them.”

– Dr Aisling Tuite, HECAT

Tuite has looked at how digital technologies can help job seekers find suitable work.

She coordinated an EU-funded project to help out-of-work people find jobs or develop new skills through a more open online system.

Called HECAT, the project ran from February 2020 through July 2023 and brought together researchers from Denmark, France, Ireland, Slovenia, Spain and Switzerland.

In recent years, many countries have brought in active labour-market policies that deploy computer-based systems to profile workers and help career counsellors target people most in need of help.

While this sounds highly targeted, Tuite said that in reality it often pushes people into employment that might be unsuitable for them and is creating job-retention troubles.

‘Our current employment systems often fail to get people to the right place – they just move people on,’ she said. ‘What people often need is individualised support or new training. We wanted to develop a product that could be as useful for people looking for work as for those supporting them.’

Ready to runHECAT’s online system combines new vacancies with career counselling and current labour-market data.

The system was tested during the project and a beta version is now available via My Labour Market and can be used in all EU countries where data is available.

It can help people figure out where there are jobs and how to be best positioned to secure them, according to Tuite.

In addition to displaying openings by location and quality, the system offers detailed information about career opportunities and labour-market trends including the kinds of jobs on the rise in particular areas and the average time it takes to find a position in a specific sector.

Tuite said feedback from participants in the test was positive.

She recalled one young female job seeker saying it had made her more confident in exploring new career paths and another who said knowing how long the average “jobs wait” would be eased the stress of hunting.

Looking ahead, Tuite hopes the HECAT researchers can demonstrate the system in governmental employment-services organisations in numerous EU countries over the coming months.

‘There is growing interest in this work from across public employment services in the EU and we’re excited,’ she said.

(This article was updated on 21 September 2023 to include a reference to Steven Dhondt’s role at TNO in the Netherlands)

Research in this article was funded by the EU.

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

In this episode, Abate flew to Denver, Colorado, to get a behind-the-scenes look at the future of recycling with Joe Castagneri, the head of AI at Amp Robotics. With Materials Recovery Facilities (MRFs) processing a staggering 25 tons of trash per hour, robotic sorting is the clear long-term solution.

Recycling is a for-profit industry. When the margins don’t make sense, the items will not be recycled. This is why Amp’s mission to use robotics and AI to bring down the cost of recycling and increase the number of items that can be sorted for recycling is so impactful.

Joe Castagneri
Joe Castagneri graduated with his Master of Science in Applied Mathematics, with an undergrad degree in Physics. While still in university, he first joined the team at Amp Robotics in 2016 where he worked on Machine Learning models to identify recyclables in video streams of Trash in Materials Recovery Facilities (MRFs). Today, he is the Head of AI at Amp Robotics where he is changing the economics of recycling through automation.

View Details

Claire chatted to Kate Devlin from King’s College London about the social and ethical implications of robotics and AI.

Kate Devlin is Reader in Artificial Intelligence & Society in the Department of Digital Humanities, King’s College London. She is an interdisciplinary computer scientist investigating how people interact with and react to technologies, both past and future. Kate is the author of Turned On: Science, Sex and Robots, which examines the ethical and social implications of technology and intimacy. She is Creative and Outreach lead for the UKRI Responsible Artificial Intelligence UK programme — an international research and innovation ecosystem for responsible AI.

View Details

Credits: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)

Did you have the chance to attend the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023) in Detroit? Here we bring you the papers that received an award this year in case you missed them. And good news: you can read all the papers because IROS on Demand is open to the public and freely available for one year from Oct 9th. Congratulations to all the winners and finalists!

IROS 2023 Best Overall and Best Student PaperWinner of the IROS 2023 Best Paper

• Autonomous Power Line Inspection with Drones via Perception-Aware MPC, by Jiaxu Xing, Giovanni Cioffi, Javier Hidalgo Carrio, Davide Scaramuzza.

Winner of the IROS 2023 Best Student Paper

• Controlling Powered Prosthesis Kinematics over Continuous Transitions Between Walk and Stair Ascent, by Shihao Cheng, Curt A. Laubscher, Robert D. Gregg.

Finalists

• Learning Contact-Based State Estimation for Assembly Tasks, by Johannes Pankert, Marco Hutter.
• Swashplateless-elevon Actuation for a Dual-rotor Tail-sitter VTOL UAV, by Nan Chen, Fanze Kong, Haotian Li, Jiayuan Liu, Ziwei Ye, Wei Xu, Fangcheng Zhu, Ximin Lyu, Fu Zhang.
• Towards Legged Locomotion on Steep Planetary Terrain, by Giorgio Valsecchi, Cedric Weibel, Hendrik Kolvenbach, Marco Hutter.
• Decentralized Swarm Trajectory Generation for LiDAR-based Aerial Tracking in Cluttered Environments, by Longji Yin, Fangcheng Zhu, Yunfan Ren, Fanze Kong, Fu Zhang.
• Open-Vocabulary Affordance Detection in 3D Point Clouds, by Toan Nguyen, Minh Nhat Vu, An Vuong, Dzung Nguyen, Thieu Vo, Ngan Le, Anh Nguyen.
• Discovering Symbolic Adaptation Algorithms from Scratch, by Stephen Kelly, Daniel Park, Xingyou Song, Mitchell McIntire, Pranav Nashikkar, Ritam Guha, Wolfgang Banzhaf, Kalyanmoy Deb, Vishnu Boddeti, Jie Tan, Esteban Real.
• Parallel cell array patterning and target cell lysis on an optoelectronic micro-well device, by Chunyuan Gan, Hongyi Xiong, Jiawei Zhao, Ao Wang, Chutian Wang, Shuzhang Liang, Jiaying Zhang, Lin Feng.
• FATROP: A Fast Constrained Optimal Control Problem Solver for Robot Trajectory Optimization and Control, by Lander Vanroye, Ajay Suresha Sathya, Joris De Schutter, Wilm Decré.
• GelSight Svelte: A Human Finger-Shaped Single-Camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing, by Jialiang Zhao, Edward Adelson.
• Shape Servoing of a Soft Object Using Fourier Series and a Physics-based Model, by Fouad Makiyeh, Francois Chaumette, Maud Marchal, Alexandre Krupa.

IROS Best Paper Award on Agri-Robotics sponsored by YANMARWinner

• Visual, Spatial, Geometric-Preserved Place Recognition for Cross-View and Cross-Modal Collaborative Perception, by Peng Gao, Jing Liang, Yu Shen, Sanghyun Son, Ming C. Lin.

Finalists

• Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles, by Connor Lee, Jonathan Gustafsson Frennert, Lu Gan, Matthew Anderson, Soon-Jo Chung.
• Relative Roughness Measurement based Real-time Speed Planning for Autonomous Vehicles on Rugged Road, by Liang Wang, Tianwei Niu, Shuai Wang, Shoukun Wang, Junzheng Wang.

IROS Best Application Paper Award sponsored by ICROSWinner

• Autonomous Robotic Drilling System for Mice Cranial Window Creation: An Evaluation with an Egg Model, by Enduo Zhao, Murilo Marques Marinho, Kanako Harada.

Finalists

• Visuo-Tactile Sensor Enabled Pneumatic Device Towards Compliant Oropharyngeal Swab Sampling, by Shoujie Li, MingShan He, Wenbo Ding, Linqi Ye, xueqian WANG, Junbo Tan, Jinqiu Yuan, Xiao-Ping Zhang.
• Improving Amputee Endurance over Activities of Daily Living with a Robotic Knee-Ankle Prosthesis: A Case Study, by Kevin Best, Curt A. Laubscher, Ross Cortino, Shihao Cheng, Robert D. Gregg.
• Dynamic hand proprioception via a wearable glove with fabric sensors, by Lily Behnke, Lina Sanchez-Botero, William Johnson, Anjali Agrawala, Rebecca Kramer-Bottiglio.
• Active Capsule System for Multiple Therapeutic Patch Delivery: Preclinical Evaluation, by Jihun Lee, Manh Cuong Hoang, Jayoung Kim, Eunho Choe, Hyeonwoo Kee, Seungun Yang, Jongoh Park, Sukho Park.

IROS Best Entertainment and Amusement Paper Award sponsored by JTCFWinner

• DoubleBee: A Hybrid Aerial-Ground Robot with Two Active Wheels, by Muqing Cao, Xinhang Xu, Shenghai Yuan, Kun Cao, Kangcheng Liu, Lihua Xie.

Finalists

• Polynomial-based Online Planning for Autonomous Drone Racing in Dynamic Environments, by Qianhao Wang, Dong Wang, Chao Xu, Alan Gao, Fei Gao.
• Bistable Tensegrity Robot with Jumping Repeatability based on Rigid Plate-shaped Compressors, by Kento Shimura, Noriyasu Iwamoto, Takuya Umedachi.

IROS Best Industrial Robotics Research for Applications sponsored by Mujin Inc.Winner

• Toward Closed-loop Additive Manufacturing: Paradigm Shift in Fabrication, Inspection, and Repair, by Manpreet Singh, Fujun Ruan, Albert Xu, Yuchen Wu, Archit Rungta, Luyuan Wang, Kevin Song, Howie Choset, Lu Li.

Finalists

• Learning Contact-Based State Estimation for Assembly Tasks, by Johannes Pankert, Marco Hutter.
• Bagging by Learning to Singulate Layers Using Interactive Perception, by Lawrence Yunliang Chen, Baiyu Shi, Roy Lin, Daniel Seita, Ayah Ahmad, Richard Cheng, Thomas Kollar, David Held, Ken Goldberg.
• Exploiting the Kinematic Redundancy of a Backdrivable Parallel Manipulator for Sensing During Physical Human-Robot Interaction, by Arda Yigit, Tan-Sy Nguyen, Clement Gosselin.

IROS Best Paper Award on Cognitive Robotics sponsored by KROSWinner

• Extracting Dynamic Navigation Goal from Natural Language Dialogue, by Lanjun Liang, Ganghui Bian, Huailin Zhao, Yanzhi Dong, Huaping Liu.

Finalists

• EasyGaze3D: Towards Effective and Flexible 3D Gaze Estimation from a Single RGB Camera, by Jinkai Li, Jianxin Yang, Yuxuan Liu, ZHEN LI, Guang-Zhong Yang, Yao Guo.
• Team Coordination on Graphs with State-Dependent Edge Cost, by Sara Oughourli, Manshi Limbu, Zechen Hu, Xuan Wang, Xuesu Xiao, Daigo Shishika.
• Is Weakly-supervised Action Segmentation Ready For Human-Robot Interaction? No, Let’s Improve It With Action-union Learning, by Fan Yang, Shigeyuki Odashima, Shochi Masui, Shan Jiang.
• Exploiting Spatio-temporal Human-object Relations using Graph Neural Networks for Human Action Recognition and 3D Motion Forecasting, by Dimitrios Lagamtzis, Fabian Schmidt, Jan Reinke Seyler, Thao Dang, Steffen Schober.

IROS Best Paper Award on Mobile Manipulation sponsored by OMRON Sinic X Corp.Winner

• A perching and tilting aerial robot for precise and versatile power tool work on vertical walls, by Roman Dautzenberg, Timo Küster, Timon Mathis, Yann Roth, Curdin Steinauer, Gabriel Käppeli, Julian Santen, Alina Arranhado, Friederike Biffar, Till Kötter, Christian Lanegger, Mike Allenspach, Roland Siegwart, Rik Bähnemann.

Finalists

• Placing by Touching: An empirical study on the importance of tactile sensing for precise object placing, by Luca Lach, Niklas Wilhelm Funk, Robert Haschke, Séverin Lemaignan, Helge Joachim Ritter, Jan Peters, Georgia Chalvatzaki.
• Efficient Object Manipulation Planning with Monte Carlo Tree Search, by Huaijiang Zhu, Avadesh Meduri, Ludovic Righetti.
• Sequential Manipulation Planning for Over-actuated UAMs, by Yao Su, Jiarui Li, Ziyuan Jiao, Meng Wang, Chi Chu, Hang Li, Yixin Zhu, Hangxin Liu.
• On the Design of Region-Avoiding Metrics for Collision-Safe Motion Generation on Riemannian Manifolds, by Holger Klein, Noémie Jaquier, Andre Meixner, Tamim Asfour.

IROS Best RoboCup Paper Award sponsored by RoboCup FederationWinner

• Sequential Neural Barriers for Scalable Dynamic Obstacle Avoidance, by Hongzhan Yu, Chiaki Hirayama, Chenning Yu, Sylvia Herbert, Sicun Gao.

Finalists

• Anytime, Anywhere: Human Arm Pose from Smartwatch Data for Ubiquitous Robot Control and Teleoperation, by Fabian Clemens Weigend, Shubham Sonawani, Drolet Michael, Heni Ben Amor.
• Effectively Rearranging Heterogeneous Objects on Cluttered Tabletops, by Kai Gao, Justin Yu, Tanay Sandeep Punjabi, Jingjin Yu.
• Prioritized Planning for Target-Oriented Manipulation via Hierarchical Stacking Relationship Prediction, by Zewen Wu, Jian Tang, Xingyu Chen, Chengzhong Ma, Xuguang Lan, Nanning Zheng.

IROS Best Paper Award on Robot Mechanisms and Design sponsored by ROBOTISWinner

• Swashplateless-elevon Actuation for a Dual-rotor Tail-sitter VTOL UAV, by Nan Chen, Fanze Kong, Haotian Li, Jiayuan Liu, Ziwei Ye, Wei Xu, Fangcheng Zhu, Ximin Lyu, Fu Zhang.

Finalists

• Hybrid Tendon and Ball Chain Continuum Robots for Enhanced Dexterity in Medical Interventions, by Giovanni Pittiglio, Margherita Mencattelli, Abdulhamit Donder, Yash Chitalia, Pierre Dupont.
• c^2: Co-design of Robots via Concurrent-Network Coupling Online and Offline Reinforcement Learning, by Ci Chen, Pingyu Xiang, Haojian Lu, Yue Wang, Rong Xiong.
• Collision-Free Reconfiguration Planning for Variable Topology Trusses using a Linking Invariant, by Alexander Spinos, Mark Yim.
• eViper: A Scalable Platform for Untethered Modular Soft Robots, by Hsin Cheng, Zhiwu Zheng, Prakhar Kumar, Wali Afridi, Ben Kim, Sigurd Wagner, Naveen Verma, James Sturm, Minjie Chen.

IROS Best Paper Award on Safety, Security, and Rescue Robotics in memory of Motohiro Kisoi sponsored by IRSIWinner

• mCLARI: A Shape-Morphing Insect-Scale Robot Capable of Omnidirectional Terrain-Adaptive Locomotion, by Heiko Dieter Kabutz, Alexander Hedrick, William Parker McDonnell, Kaushik Jayaram.

Finalists

• Towards Legged Locomotion on Steep, Planetary Terrain, by Giorgio Valsecchi, Cedric Weibel, Hendrik Kolvenbach, Marco Hutter.
• Global Localization in Unstructured Environments using Semantic Object Maps Built from Various Viewpoints, by Jacqueline Ankenbauer, Parker C. Lusk, Jonathan How.
• EELS: Towards Autonomous Mobility in Extreme Environments with a Novel Large-Scale Screw Driven Snake Robot, by Rohan Thakker, Michael Paton, Marlin Polo Strub, Michael Swan, Guglielmo Daddi, Rob Royce, Matthew Gildner, Tiago Vaquero, Phillipe Tosi, Marcel Veismann, Peter Gavrilov, Eloise Marteau, Joseph Bowkett, Daniel Loret de Mola Lemus, Yashwanth Kumar Nakka, Benjamin Hockman, Andrew Orekhov, Tristan Hasseler, Carl Leake, Benjamin Nuernberger, Pedro F. Proença, William Reid, William Talbot, Nikola Georgiev, Torkom Pailevanian, Avak Archanian, Eric Ambrose, Jay Jasper, Rachel Etheredge, Christiahn Roman, Daniel S Levine, Kyohei Otsu, Hovhannes Melikyan, Richard Rieber, Kalind Carpenter, Jeremy Nash, Abhinandan Jain, Lori Shiraishi, Ali-akbar Agha-mohammadi, Matthew Travers, Howie Choset, Joel Burdick, Masahiro Ono.
• Multi-IMU Proprioceptive Odometry for Legged Robots, by Shuo Yang, Zixin Zhang, Benjamin Bokser, Zachary Manchester.

View Details

The International Conference on Intelligent Robots and Systems (IROS) showcases leading-edge research in robotics. IROS was held in Detroit MI Oct 1-5 and not only showcased research but the latest commercialization in robotics, particularly robotics providers selling into robotics for research or as part of the hardware/software stack. The conference focuses on future directions in robotics, and the latest approaches, designs, and outcomes. It also provides an opportunity to network with the world’s leading roboticists.

Highlights included seeing Silicon Valley Robotics members; Foxglove, Hello Robot, Anyware Robotics and Tangram Vision, also Open Robotics and Intrinsic talking up ROS 2 and the upcoming ROSCon 23. Intrinsic sponsored a ROS/IROS meetup and Clearpath Robotics sponsored the Diversity Cocktails event. OhmniLabs sponsored 3 telepresence robots which were in constant demand touring the expo, the competition floor and the poster sessions. I also met Sol Robotics from the Bay Area, which has quite a unique robot arm structure, that’s super stable with the ability to carry a lot of weight.

There were plenty of rolling and roaming robots, like this Diablo from Direct Drive Tech (world’s first Direct-Drive Self-Balancing Wheeled-Leg Robot), also from Deep Robotics, Unitree Robotics, Fourier Intelligence, Hebi and Westwood Robotics. Also the other legged one, and the rolling robots – Clearpath, Otto, Husarion, Hebi and more. Although they weren’t on the Expo Floor, the Disney keynote session was another highlight with a live robot demo on stage,

And Franka Emika fans will be pleased to hear that not only did they win a ‘best paper’ award, but that the eminent demise of the company is much overstated. It’s a German thing. There are many investors/purchasers lined up to keep the company going while they restructure. And watch out for Psyonics! Psyonics’ smart ability hands and arms, world’s first touch sensing bionic arms, are being used by Apptronik (humanoid for NASA) as well as for people with disabilities.

IROS Exhibitor gallery

Full list of IROS Exhibitors is here.

View Details

MIT researchers have developed a camera-based touch sensor that is long, curved, and shaped like a human finger. Their device, which provides high-resolution tactile sensing over a large area, could enable a robotic hand to perform multiple types of grasps. Image: Courtesy of the researchers

By Adam Zewe | MIT News

Imagine grasping a heavy object, like a pipe wrench, with one hand. You would likely grab the wrench using your entire fingers, not just your fingertips. Sensory receptors in your skin, which run along the entire length of each finger, would send information to your brain about the tool you are grasping.

In a robotic hand, tactile sensors that use cameras to obtain information about grasped objects are small and flat, so they are often located in the fingertips. These robots, in turn, use only their fingertips to grasp objects, typically with a pinching motion. This limits the manipulation tasks they can perform.

MIT researchers have developed a camera-based touch sensor that is long, curved, and shaped like a human finger. Their device provides high-resolution tactile sensing over a large area. The sensor, called the GelSight Svelte, uses two mirrors to reflect and refract light so that one camera, located in the base of the sensor, can see along the entire finger’s length.

In addition, the researchers built the finger-shaped sensor with a flexible backbone. By measuring how the backbone bends when the finger touches an object, they can estimate the force being placed on the sensor.

They used GelSight Svelte sensors to produce a robotic hand that was able to grasp a heavy object like a human would, using the entire sensing area of all three of its fingers. The hand could also perform the same pinch grasps common to traditional robotic grippers.

This gif shows a robotic hand that incorporates three, finger-shaped GelSight Svelte sensors. The sensors, which provide high-resolution tactile sensing over a large area, enable the hand to perform multiple grasps, including pinch grasps that use only the fingertips and a power grasp that uses the entire sensing area of all three fingers. Credit: Courtesy of the researchers

“Because our new sensor is human finger-shaped, we can use it to do different types of grasps for different tasks, instead of using pinch grasps for everything. There’s only so much you can do with a parallel jaw gripper. Our sensor really opens up some new possibilities on different manipulation tasks we could do with robots,” says Alan (Jialiang) Zhao, a mechanical engineering graduate student and lead author of a paper on GelSight Svelte.

Zhao wrote the paper with senior author Edward Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research will be presented at the IEEE Conference on Intelligent Robots and Systems.

Mirror mirrorCameras used in tactile sensors are limited by their size, the focal distance of their lenses, and their viewing angles. Therefore, these tactile sensors tend to be small and flat, which confines them to a robot’s fingertips.

With a longer sensing area, one that more closely resembles a human finger, the camera would need to sit farther from the sensing surface to see the entire area. This is particularly challenging due to size and shape restrictions of a robotic gripper.

Zhao and Adelson solved this problem using two mirrors that reflect and refract light toward a single camera located at the base of the finger.

GelSight Svelte incorporates one flat, angled mirror that sits across from the camera and one long, curved mirror that sits along the back of the sensor. These mirrors redistribute light rays from the camera in such a way that the camera can see the along the entire finger’s length.

To optimize the shape, angle, and curvature of the mirrors, the researchers designed software to simulate reflection and refraction of light.

“With this software, we can easily play around with where the mirrors are located and how they are curved to get a sense of how well the image will look after we actually make the sensor,” Zhao explains.

The mirrors, camera, and two sets of LEDs for illumination are attached to a plastic backbone and encased in a flexible skin made from silicone gel. The camera views the back of the skin from the inside; based on the deformation, it can see where contact occurs and measure the geometry of the object’s contact surface.

A breakdown of the components that make up the finger-like touch sensor. Image: Courtesy of the researchers

In addition, the red and green LED arrays give a sense of how deeply the gel is being pressed down when an object is grasped, due to the saturation of color at different locations on the sensor.

The researchers can use this color saturation information to reconstruct a 3D depth image of the object being grasped.

The sensor’s plastic backbone enables it to determine proprioceptive information, such as the twisting torques applied to the finger. The backbone bends and flexes when an object is grasped. The researchers use machine learning to estimate how much force is being applied to the sensor, based on these backbone deformations.

However, combining these elements into a working sensor was no easy task, Zhao says.

“Making sure you have the correct curvature for the mirror to match what we have in simulation is pretty challenging. Plus, I realized there are some kinds of superglue that inhibit the curing of silicon. It took a lot of experiments to make a sensor that actually works,” he adds.

Versatile graspingOnce they had perfected the design, the researchers tested the GelSight Svelte by pressing objects, like a screw, to different locations on the sensor to check image clarity and see how well it could determine the shape of the object.

They also used three sensors to build a GelSight Svelte hand that can perform multiple grasps, including a pinch grasp, lateral pinch grasp, and a power grasp that uses the entire sensing area of the three fingers. Most robotic hands, which are shaped like parallel jaw drippers, can only perform pinch grasps.

A three-finger power grasp enables a robotic hand to hold a heavier object more stably. However, pinch grasps are still useful when an object is very small. Being able to perform both types of grasps with one hand would give a robot more versatility, he says.

Moving forward, the researchers plan to enhance the GelSight Svelte so the sensor is articulated and can bend at the joints, more like a human finger.

“Optical-tactile finger sensors allow robots to use inexpensive cameras to collect high-resolution images of surface contact, and by observing the deformation of a flexible surface the robot estimates the contact shape and forces applied. This work represents an advancement on the GelSight finger design, with improvements in full-finger coverage and the ability to approximate bending deflection torques using image differences and machine learning,” says Monroe Kennedy III, assistant professor of mechanical engineering at Stanford University, who was not involved with this research. “Improving a robot’s sense of touch to approach human ability is a necessity and perhaps the catalyst problem for developing robots capable of working on complex, dexterous tasks.”

This research is supported, in part, by the Toyota Research Institute.

• PAPER – GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing. Jialiang Zhao and Edward H. Adelson. arXive.
• PAPER – GelSight Svelte Hand: A Three-finger, Two-DoF, Tactile-rich, Low-cost Robot Hand for Dexterous Manipulation. Jialiang Zhao and Edward H. Adelson. arXive.

View Details

Claire chatted to Guillaume Doisy from Dexory about autonomous warehouse robots, mobile robotics, and software.

Guillaume Doisy is the current Lead Systems Architect at Dexory, the leading UK robotics and AI company building state-of-the-art fully autonomous mobile robots for use in warehouses. Guillaume has a wide range of responsibilities including building the robots ability to function autonomously. The French native has significant expertise in the field of autonomous mobile robots having begun his career at French start-up Wyca as Chief Robotics Officer. Guillaume is also a long-term contributor to ROS (Robotic Operating Software), the open-source robotics software project.

View Details

In celebration of the launch of International Women in Robotics Day, the Women in Robotics organization is proud to release another “50 women in robotics you need to know about” collection of stories. With a growing robotics industry there are many opportunities for everyone to get involved. This is why we showcase the wide range of roles that women play in robotics today.

Since 2012, the Women in Robotics organization has released a list of women building the future in robotics. The list has covered all ages, career stages, types of occupation and experience. We’ve featured more than 350 women already and we’ve shown that women have always been working in the robotics industry, in the earliest robotics research labs and companies, although those stories have often been forgotten.

This year’s collection includes Nancy Cornelius, co-founder of Boston Dynamics and the first engineer hired. Cornelius remained an integral part of Boston Dynamics until the company was sold to Google in 2013. Vandi Verma is the head of NASA’s rover (robot) program. Joanna Buttler is the head of the Global Autonomous Technology Group for Daimler Truck. And Whitney Rockley founded a venture capital company investing exclusively in ‘industrial internet’ companies like Clearpath Robotics.

For the first time, we feature an Indigenous (Ojibwe) American roboticist, Danielle Boyer. Boyer started a non-profit The STEAM Connection to combat the difficulties that many kids have getting access to robotics. She created an affordable robot kit that’s been distributed to thousands of students, and is proudest of the SKOBOT project. Personalized robots that keep culture and language traditions alive. Boyer epitomizes the motto “Building the Future”.

We also try to feature women from all regions of the world and this year’s collection represents Nigeria, India, China, Australia, Japan, Switzerland, Croatia, Korea, Denmark, Singapore, Italy, Romania, United States, Sweden, Spain, Canada, the UK, Israel, Austria, Belgium, Mexico, Argentina and Brazil. There is an active Latinx community in Women in Robotics engaged in translating more robotics information into Spanish, hoping to create more connections between the global robotics community and the roboticists, and potential roboticists, of Latin America.

There have always been women doing great things in robotics! And we’re pleased to present another collection of strong female role models for young and upcoming roboticists (of any gender).

You can also join in the Women in Robotics celebrations today and throughout October, with events listed on the women in robotics site, like Diversity Cocktails at the IROS conference in Detroit, or the launch of the Los Angeles women in robotics chapter. Women in Robotics is a global community organization for women and non-binary people working in robotics and those who’d like to work in robotics. Learn more at https://womeninrobotics.org

Join our events, host your own events, share our celebration on social media!

View Details

The new World Robotics report recorded 553,052 industrial robot installations in factories around the world – a growth rate of 5% in 2022, year-on-year. By region, 73% of all newly deployed robots were installed in Asia, 15% in Europe and 10% in the Americas.

“The world record of 500,000 units was exceeded for the second year in succession,” says Marina Bill, President of the International Federation of Robotics. “In 2023 the industrial robot market is expected to grow by 7% to more than 590,000 units worldwide.”

Asia, Europe and the Americas – overviewChina is by far the world´s largest market. In 2022, annual installations of 290,258 units replaced the previous record of 2021 by growth of 5%. This latest gain is remarkable since it even tops the 2021 result that was a 57% jump compared to 2020. To serve this dynamic market, domestic and international robot suppliers have established production plants in China and continuously increased capacity. On average, annual robot installations have grown by 13% each year (2017-2022).

Robot installations in Japan were up by 9% to 50,413 units, exceeding the pre-pandemic level of 49,908 units in 2019. The peak level remains at 55,240 units in 2018. The country ranks second to China in size of market for industrial robots. Annual installations gained 2% on average per year (2017-2022). Japan is the world´s predominant robot manufacturing country with a market share of 46% of the global robot production.

The market in the Republic of Korea rose by 1% – installations reached 31,716 units in 2022. This was the second year of marginal growth, following four years of declining installation figures. The Republic of Korea remains the fourth largest robot market in the world, following the United States, Japan, and China.

EuropeThe European Union remains the world´s second largest market (70,781 units; +5%) in 2022. Germany is one of the top five adopters worldwide with a market share of 36% within the EU. Germany´s installations went down by 1% to 25,636 units. Italy follows with a market share of 16% within the EU – installations grew by 8% to 11,475 units. The third largest EU market, France, recorded a regional market share of 10% and gained 13%, installing 7,380 units in 2022.

In the post-Brexit United Kingdom, industrial robot installations were up by 3% to 2,534 units in 2022. This is less than a tenth of Germany´s sales.

The AmericasIn the Americas, installations were up 8% to 56,053 units in 2022, surpassing the 2018 peak level (55,212 units). The United States, the largest regional market, accounted for 71% of the installations in the Americas in 2022. Robot installations were up by 10% to 39,576 units. This was just shy of the peak level of 40,373 units achieved in 2018. The main growth driver was the automotive industry that displayed surging installations by +47% (14,472 units). The share of the automotive industry has now grown back to 37%, followed by the metal and machinery industry (3,900 units) and the electrical/electronics industry (3,732 units).

The two other major markets are Mexico – here installations grew by 13% (6,000 units) – and Canada, where demand dropped by 24% (3,223 units). This was the result of lower demand from the automotive industry – the strongest adopter.

Brazil is an important production site for motor vehicles and automotive parts: The International Organization of Motor Vehicle Manufacturers (OICA) reports an output of 2.4 million vehicles in 2022. This shows the huge potential for automation in the country. Annual installation counts grew rather slowly with cyclical ups and downs. In 2022, 1,858 robots were installed. This was 4% more than in the previous year.

OutlookThe year 2023 will be characterized by a slowdown of the global economic growth. Robot installations in 2023 are not expected to follow this pattern. There is no indication that the overall long-term growth trend will come to an end soon: rather the contrary will be the case. The mark of 600,000 units installed per year worldwide is expected to be reached in 2024.

View Details

Credits: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023).

The 2023 EEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023) kicks off today at the Huntington Place in Detroit, Michigan. This year’s theme, “The Next Generation of Robotics,” is a call to the young and senior researchers to create a forum where the past, present, and future of robotics converge.

The program of IROS 2023 is a blend of theoretical insights and practical demonstrations, designed to foster a culture of innovation and collaboration. Among the highlights are the plenary and keynote talks by eminent personalities in the field of robotics.

Plenaries and keynotes

On the plenary front, Marcie O’Malley from Rice University will delve into the realm of robots that teach and learn with a human touch. Yuto Nakanishi of GITAI, Japan, will share his insights on the challenges of developing space robots for building a moonbase. Matt Johnson-Roberson from Carnegie Mellon University will explore the shared history and convergent future of AI and Robotics.

The keynote sessions are equally thought-provoking. On Monday, October 2nd, Sven Behnke from the University of Bonn, Germany, will discuss the transition from intuitive immersive telepresence systems to conscious service robots, while Michelle Johnson from the University of Pennsylvania, USA, will talk about the journey towards more inclusive rehabilitation robots. Rebecca Kramer-Bottiglio from Yale University, USA, will also share insights on shape-shifting soft robots that adapt to changing tasks and environments.

On Tuesday, October 3rd, Kostas Alexis from the Norwegian University of Science and Technology, Norway, will share experiences from the DARPA Subterranean Challenge focusing on resilient robotic autonomy. Serena Ivaldi from Inria, France, will discuss the transition from humanoids to exoskeletons, aiming at assisting and collaborating with humans. Mario Santillo from Ford Motor Company, USA, will provide a glimpse into the future of manufacturing automation.

The series continues on Wednesday, October 4th, with Moritz Bächer (Switzerland) and Morgan Pope (USA) from Disney Research discussing the design and control of expressive robotic characters. Tetsuya Ogata from Waseda University/AIST, Japan, will delve into deep predictive learning in robotics, optimizing models for adaptive perception and action. Lastly, Teresa Vidal-Calleja from the University of Technology Sydney, Australia, will talk about empowering robots with continuous space and time representations.

Competitions

The competitions segment of IROS 2023 will be a space for innovation and creativity. The Functional Fashion competition invites teams to design and demonstrate robotic clothing that is as aesthetically pleasing as it is functional. The F1/10 Autonomous Racing challenges participants to build a 1:10 scaled autonomous race car and compete in minimizing lap time while avoiding crashes. The Soft Robotics Balloon Robots competition encourages the creation of locomoting and swimming soft robots using balloons as a substrate, exploring rapid design and deployment of soft robotic structures.

Technical programme & demonstrations

The technical sessions and workshops/tutorials at IROS 2023 are designed to foster a rich exchange of ideas among the attendees. These sessions will feature presentations on cutting-edge research and innovative projects from across the globe, providing a platform for researchers to share their findings, receive feedback, and engage in meaningful discussions. In additions, the demonstrations segment will bring theories to life as participants showcase their working prototypes and models, offering a tangible glimpse into the advancements in robotics.

Participate in IROS 2023 remotely with Ohmni telepresence robotsIf you are unable to participate in the conference in person, Ohmnilabs has provided three of their Ohmni telepresence robots to facilitate participation in the conference virtually. The telepresence robots will be active from October 2-4 from 9:00 a.m.- 6:00 p.m. EDT. You can secure a time slot in advance using this link.

The telepresence robots will allow you to:

• Explore the exhibit hall and speak with exhibitors
• Interact with authors and other attendees during interactive poster sessions
• Attend a plenary or keynote presentation

You can check in real-time here to see if any of the robots are available throughout the day.

Watch out our blog during the following days for updates and results from the best paper awards. And enjoy IROS 2023!

View Details

Claire chatted to Sara Adela Abad Guaman from University College London about adaptable robots inspired by nature.

Sara Adela Abad Guaman is a Lecturer at University College London’s Mechanical Engineering Department. She is also the head of the Adaptable Robotics Lab. Inspired by biological organisms, Sara aims to develop robots and mechanical systems with enhanced adaptability to variable environmental conditions. Her vision is to use bioinspiration and morphological computation to address global challenges such as climate change, biodiversity loss, and sustainability.

View Details

This guide to ‘colliding opposite disciplines with your research’ is intended to help students and researchers, or indeed anyone who might otherwise be looking for some ideas on how to approach research or methods for designing concepts and solutions, to broaden their thinking and approach to research. This guide is mainly focused on the disciplines of science and engineering with the idea of collaborating with other distinct disciplines. However, the overall principles remain for any multidisciplinary research.

The guide is written into three different sections;

1. Is it all just hot STEAM?
2. When worlds collide.
3. The common goal – how can I develop my multidisciplinary research?

With the assistance of this guide, it will help to open new ways of thinking about research, highlight the ‘unseen’ benefits of multidisciplinary approaches to research and how they can be extremely advantageous and can lend for an optimal delivery. It will help you to contemplate how, when, and why you should open up your research to other disciplines.

Is it all just hot STEAM?If we think of the Arts and Science then I think, for the most part, that people would think of them as opposites. People are either ‘arty’ or ‘science-y’. A large factor in this thinking may come from the fact that people are seen as left-brained or right-brained, where one side of the brain is dominant. Left-brained thinkers are said to be methodical and analytical, whereas right-brained thinkers are said to be creative or artistic. What should happen, though, if you were able to work across these two separated sides and create whilst you analyse?

Do we need to get out of this thought, that art and science don’t belong together? I would firmly argue that we do.

The acronym STEM is widely known as Science, Technology, Engineering and Mathematics. However, another acronym, possibly less well known, is STEAM. STEAM is Science, Technology, Engineering, (liberal) Arts and Mathematics. This represents all aspects of art such as drama, music, design, media and visual arts. STEM primarily focuses on scientific concepts, whilst STEAM investigates the same concepts, but does this through investigation and problem-based learning methods used in an imaginative process.

The application of the arts to science is not a new practice Leonardo DaVinci is an early example of someone using STEAM to make discoveries and explain them to several generations.

There are many advantages to applying the arts to science and engineering. For example, would increasing application of the arts to science and engineering make more young people want to do science and engineering as it looks visually more attractive and significant? Could it help them develop a love for the STEM subjects and support them to seeing it as being more relatable than a Bunsen burner in a school laboratory.

A prime, and very recent, example of STEAM being applied was the crewed Space-X launch of the Dragon capsule in 2020. This launch represents the essence of advanced technology that is both on the forefront of science and engineering development as well technological aesthetics. From the design of the sleek logos, through to the futuristic spacesuits and even continuing onto the matt black launch platform, it was clear throughout this launch that every single detail had been considered.

Some may argue that by adding this artistic touch to technology that the ‘nitty-gritty science’ of the design and aesthetic becomes lost. If something doesn’t ‘look’ complicated and complex can it really be advanced or sophisticated? Well yes! Have you ever heard the saying “when someone makes something look simple, they have spent hours perfecting it” Take for example the space suits and touch screen controls of the Dragon SpaceX launch. The suits look like they were designed for a film set of Hollywood’s renditions of Space travel. A spacesuit without oxygen inlets, pressurized helmets and fitted to individual body contours.

To the layman, these may simply look like pleasant visuals. Though, to the trained eye and relevantly knowledgeable mind the pure fact that these two components of the flight look so streamline and simplistic not only nods to but reinforces that all aspects of the flight was saturated in superior engineering from hundreds of magnificent minds. That is the beauty of STEAM.

When worlds collideThere are several advantages to multidisciplinary research. Now more than before, there has been focus on research becoming more multidisciplinary. As the world is in the fourth industry revolution (Industry 4) and with constant and significant advances in technology and AI there is increased necessity for research to meet complex and substantial scientific and engineering global challenges. Consider, a real-world problem, either one you know something about or one you are researching. Completing all aspects of this issue and its application you will notice that, repeatedly, it cannot be confined to one single discipline.

A multidisciplinary environment in research allows for different theories, methodologies, modes of thinking (convergent, divergent and lateral) and perspectives to come together for one common goal and purpose. The beauty of multidisciplinary research sits within this divergence of thinking, approaches, and theories which provides a much broader context to create innovate and bespoke discoveries and solutions.

In research, it can often be the case that students or academics end up working in quite a niche area of research, which of course has great advantages of its own as one can become a leading expert in a particular field. However, when a specific piece of research is presented that needs expert knowledge and experience from another field or discipline it can be difficult for an individual to become skilled or versed enough (often within pressing timelines) to lead on that area of the work. Here, a multidisciplinary environment/team will allow for contributions and skilled knowledge from other disciplines to have input without all researchers having to masters each other’s skills and knowledge but to only understand.

Often, an effective way to further show value and demonstrate a concept is to lead by example…

Consider the discipline of robotics. A robot represents a wide array of disciplines.

In a true multidisciplinary approach, there is great research capability in designing and building new robotic systems, that offers the ability to apply bespoke robotic based solutions to a range of applications.

In the case of social and healthcare robotics. Here, robotics represents aspects of electrical/mechanical engineering, material science, psychology, and medicine. In these environments, robots are much more than circuitry and AI. Other disciplines come from the end user requirements, the operational environments, and bespoke requirements/purposes.

Design in robotics is something that is often overlooked. However, it is extremely important in the creation of robotic solutions for both end users and client requirements. The physical appearance of a robot can affect presumptions and expectations of how a robotic system should or will perform.

Aesthetics, more than one is conscious of, influences aspects of our lives, and the decisions we make. The collaboration of the arts/design and robotics can be particularly effective towards increasing trust in robots. Establishing human-robot interaction (HRI) trust is especially pertinent where robots are being used in individual personal and healthcare roles.

For several years, it has been recognised and understood that trust is a crucial aspect of effective human-robot interaction for social robots as it closes the discipline gap between human psychology and artificial intelligence (AI).

A robot’s physical appearance can positively or negatively affect a human’s interaction with a robot. Just as humans can make decision upon first meeting someone, a human can make a similar decision based on the first impression when interacting with a robot for the first time. As a result, co-design improves the engagement and the quality of the interaction between (multidisciplinary) researchers and the end user. In the cases of healthcare or surgical robotics, it should be instinctive for a researcher to involve a person from a medical background, such as surgeons and/or medical device developers, to co-create a robotic solution fitting to the healthcare issue. To create ‘sightless’ to the knowledge and experience of the real life reality and important key issues that must be considered and adopted would be developing from a position of being completely ignorant to the challenge and effective solution.

Sustainability is a major current area of research, sustainability for the world and sustainability for technology research and development. Considering sustainability in robotic technology development the factors that should be considered, for example, are the choice & quantity of materials, the possibility of using recycled materials (e.g. in soft robotics) and effective design to limit single use robotics, or inefficient processes. This research is a topic of a truly multidisciplinary nature. How can this be considered or deciphered? It can be broken down in the resulting way;

1. Considerations – material choices, quantity of materials, effective & efficient design.
2. Following considerations – where do the materials come from, how does the technology affect the environment and society.
3. Draw out research points – lithium batteries, mining, supporting jobs roles or taking job roles, damage to environment, moving local communities.
4. What disciplines do we need? – material scientists, chemical scientists, robotics engineers, policy makers, lawyers, sociologists, economists.

The common goal – how can I develop my multidisciplinary research?A multidisciplinary research environment may be outside of your established or traditional research approaches or considerations. If so, there may be several questions that immediately come to mind about working in these types of collaborations. Such as how do you put together a multidisciplinary research group? Will the terminology and language across our disciplines be the same? How will other disciplines approach the problem – ‘will there be too many cooks’?

It is common that with change, new methodologies and approaches to working (especially if you are set in a certain way of doing things over many years), for there to be some initial challenges and a period of adaptation.

If you are interested to work in or create a multidisciplinary team here are some tips for developing this research approach.

1. Identify and acknowledge your dominant discipline/perspective (and all that it encompasses) – this can seem like an obvious point to make and think. However, considering your discipline will make you think of the fields that sit with it and this will help to identity disciplines and fields that are completely out with your area. As well as groups or researchers that do not sit within your department or similarly researchers that may be in another group within the department that you had not considered working with before.
2. Consider the research you are conducting. What are the underlying theories? Where are the natural overlaps? This can create a research ecosystem that will help you identify areas that you and others can co-create within. As a brief example, physics provides the fundamental basis for biology.
3. Identify the areas of knowledge are you lacking in? What areas of knowledge do you need strengthened? Where are the gaps in your research? This will help you to identify what type of expertise you need and where to get this from.
4. Next, identify who are the end users, what are the applications? This will help you think of the bigger picture of your research and what expertise should have input in the work. For example, in the case of healthcare, surgeons should have input in medical device robotics or in the instance of assisted living the end user must have input of information about their requirements and daily living situation.
5. Lastly, make sure you know what you are talking about. Put together a short brief of the purpose of the work and an outline answering these key indicators listed here. Ensure this conveys the purpose and end goal of the work, the gaps and where the other disciplines can add value. Identify, the group or induvial person you think could create a beneficial multidisciplinary team and contact them to present this information to them. Of course, in certain circumstances this can lead to the development of research grants!

Recognise that there are certain points to consider such as,

1. When collaborating with people from other disciplines there can be initial hurdles to overcome with how easy it is to convey your ideas to them, the languages that speak in and the ways they communicate. Take time to verse yourself in others ways of doing things and their language and methods on communicating.
2. Other disciplines may not work in a factually driven way, and it could be more a creative/holistic view of thinking. Be open minded, be open to adapting to new ways of doing things. This is advantageous to you too!
3. It may take some brainstorming sessions and design workshops (for instances) to get some momentum going in the work. However, take time to reflect on what has been done so far and always move forward with the same purpose and goal. Remember there was a reason that you created this team. Reflect on this.

Lastly, do not let these considerations stop or hinder your ideas of working in a multidisciplinary environment. There is so much to learn in these types of research teams, and it is always interesting, it is guaranteed. There is no research quite like the output form a team that is not confined into one discipline.

So, the next time you are designing, creating, or innovating, consider; am I letting off enough STEAM and are worlds colliding?

This work by Dr Karen Donaldson is licensed under a Creative Commons Attribution licence 4.0.

View Details

Source: OpenAI’s DALL·E 2 with prompt “a hyperrealistic picture of a robot reading the news on a laptop at a coffee shop”

Welcome to the 5th edition of Robo-Insight, a robotics news update! In this post, we are excited to share a range of new advancements in the field and highlight robots’ progress in areas like human-robot interaction, agile movement, enhanced training methods, soft robotics, brain surgery, medical navigation, and ecological research.

New tools for human-robot interactionIn the realm of human-robot interactions, researchers from around Europe have developed a new tool called HEUROBOX to assess interactions. HEUROBOX offers 84 basic and 228 advanced heuristics for evaluating various aspects of human-robot interaction, such as safety, ergonomics, functionality, and interfaces. It places a strong emphasis on human-centered design, addressing the vital connection between technology and human factors. This tool aims to facilitate seamless collaboration between humans and robots in industrial settings by ensuring robots align with human capabilities and needs, emphasizing productivity and well-being.

Approach for creating a novel method to assess Human-Robot Interaction (HRI) heuristics. Source.

Innovations for enhanced control in agile roboticsShifting our focus to agile robots, researchers from Zhejiang University have designed a cable-driven snake-like robot for efficient motion in confined spaces. This robot utilizes force transducers and angle sensors to achieve precise dual-loop control. By combining pose feedback from angle sensors and force feedback from transducers, this control strategy enhances the robot’s accuracy and ensures cable force and stiffness, guaranteeing stability and reliability during motion. This innovation has significant potential for various applications, including minimally invasive surgery, nuclear waste handling, in-space inspections, and search and rescue operations in complex environments. The robot’s design and control strategy promises advancements in high-precision robotic systems for engineering applications.

CSR Overview. Source.

Better training methods in agile roboticsKeeping within the field of agile robotics, researchers from the University of Zurich have recently pushed the boundaries of this type of robots, focusing on the pivotal role of control systems within them. Their investigation pitted two key methodologies against each other: model-based optimal control (OC) and reinforcement learning (RL). Surprisingly, RL, which enables robots to learn through trial and error, triumphed in a demanding real-world test: autonomous drone racing. Not only did RL outperform but surpassed human capabilities, with the agile drone achieving an astonishing peak acceleration, exceeding 12 times gravitational acceleration, and a remarkable top speed of 108 kilometers per hour. These results illuminate the promising future of agile robotics, where learning-centric approaches like RL pave the way for more efficient control and performance in diverse applications.

The visual progression shows a fast-racing drone being controlled by the RL strategy over time. Source.

New strong and stiff soft robotsChanging our focus to the world of soft robotics, recently researchers from Kangwon National University have presented a soft gripper robot with the ability to vary its stiffness, addressing a major challenge in the field of soft robotics. Unlike complex designs, this gripper achieves stiffness variation through a straightforward mechanism involving pneumatic control and tendons actuated by stepper motors. This innovation allows the gripper to adapt to objects of various shapes, sizes, and weights, expanding its potential applications. The study demonstrates that this gripper can increase its stiffness by up to 145% and handle weights of up to 2.075 kg. Soft robotics, inspired by natural organisms, holds promise in healthcare, manufacturing, exploration, and other fields, and this research contributes to its advancement.

CAD design of gripper mechanism. Source.

Enhanced brain surgery robotsTurning our focus to the medical robotics world, researchers from Harvard Medical School have developed a robotic device poised to enhance neurosurgery by making it less invasive. The team introduced a novel two-armed joystick-controlled endoscopic robot designed to mimic the dexterity of open surgery but with smaller incisions. This innovation was put to the test in the context of brain tumor resection, a typically invasive procedure. Compared to conventional manual endoscopic tools, the robot offered greater access to the surgical site, enabling bimanual tasks without brain tissue compression, and often completing tasks more swiftly. These findings open the door to the potential transformation of traditionally open brain surgeries into less invasive endoscopic procedures.

Robotic tube arms with varying degrees of stiffness. Source.

An advanced robotics needleAlong the same lines as medical robotics advancements, a team of researchers led by Professor Ron Alterovitz at the University of North Carolina at Chapel Hill has developed an autonomous robotic needle designed to navigate through intricate lung tissue while avoiding obstacles and important lung structures. The needle uses AI and computer vision to autonomously travel through living tissue, making it a potentially valuable tool for precise medical procedures like biopsies and targeted drug delivery. This development represents a significant step in the field of medical robotics, offering improved accuracy and safety in minimally invasive procedures. The researchers plan to further refine the technology and explore additional medical applications.

The robotic needle emerging from a bronchoscope. Source.

Robots could bee the key to ecological researchFinally, in the ecological field, robotics researchers from Durham University are teaming up with experts from various disciplines to investigate how animals are adapting to ecological challenges, with the aim of mitigating global biodiversity loss. Leading the RoboRoyale project, Dr. Farshad Arvin combines miniature robotics, artificial intelligence, and machine learning to develop robotic bees. These robotic bees are designed to interact with honeybee queens, enhancing their egg-laying and pheromone production, which influences hive behavior. This unique project focuses exclusively on queen bees, using a multi-robot system that learns over time how to optimize their well-being. Simultaneously, the MammalWeb project collects camera trap images to monitor the habits and behaviors of UK mammals, addressing the impact of climate change and human activities on biodiversity. These initiatives represent groundbreaking contributions from the robotics community to ecological research.

A robotic development device. Source.

The continuous evolution across various sectors underscores the adaptable and consistently progressing nature of robotics technology, revealing new opportunities for its integration into diverse industries. The gradual growth in the field of robotics reflects sustained dedication and provides insight into the potential implications of these developments for the future.

Sources:

1. Apraiz, A., Mulet Alberola, J. A., Lasa, G., Mazmela, M., & Nguyen, H. N. (2023, August 16). Development of a new set of heuristics for the evaluation of human-robot interaction in industrial settings: Heuristics Robots experience (HEUROBOX). Frontiers.
2. Xu, X., Wang, C., Xie, H., Wang, C., & Yang, H. (2023, September 4). Dual-loop control of cable-driven snake-like robots. MDPI.
3. Song, Y., Romero, A., Matthias Müller, Koltun, V., & Davide Scaramuzza. (2023). Reaching the limit in autonomous racing: Optimal control versus reinforcement learning. Science Robotics, 8(82).
4. Mawah, S. C., & Park, Y.-J. (2023, September 11). Tendon-driven variable-stiffness pneumatic soft gripper robot. MDPI.
5. Price, K., Peine, J., Mencattelli, M., Yash Chitalia, Pu, D., Looi, T., Stone, S., Drake, J. M., & Dupont, P. E. (2023). Using robotics to move a neurosurgeon’s hands to the tip of their endoscope. Science Robotics, 8(82).
6. Autonomous Medical Robot Successfully Steers Needles Through Living Tissue. (n.d.). Computer Science. Retrieved September 23, 2023
7. University, D. (n.d.). Computer Science research to build robotic bees and monitor mammals – Durham University. Www.durham.ac.uk. Retrieved September 23, 2023‌

View Details

Scientists at the Max Planck Institute for Intelligent Systems in Stuttgart have developed a soft robotic tool that promises to one day transform minimally invasive endovascular surgery. The two-part magnetic tool can help to visualise in real time the fine morphological details of partial vascular blockages such as stenoses, even in the narrowest and most curved vessels. It can also find its way through severe blockages such as chronic total occlusions. This tool could one day take the perception of endovascular medical devices a step further.

Intravascular imaging techniques and microcatheter procedures are becoming ever more advanced, revolutionizing the diagnosis and treatment of many diseases. However, current methods often fail to accurately detect the fine features of vascular disease, such as those seen from within occluded vessels, due to limitations such as uneven contrast agent diffusion and difficulty in safely accessing occluded vessels. Such limitations can delay rapid intervention and treatment of a patient.

Scientists at the Max Planck Institute for Intelligent Systems in Stuttgart have looked at this problem. They have leveraged the concepts of soft robotics and microfabrication to develop a miniature soft magnetic tool that looks like a very slim eel. This tool may one day take the perception capabilities of endovascular devices one step further. In a paper and in a video, the team shows how the tool, which is propelled forward by the blood flow, travels through the narrowest artificial vessels – whether there is a sharp bend, curve, or obstacle.

When the tool reaches an occlusion like a partially blocked artery, it performs a wave-like deformation given the external magnetic field (more on that below). Then, the deformed soft body will be gently in contact with the surrounding occluded structures. Lastly, the real-time shapes of the device when we retract it will ‘visualize’ the morphological details inside the vessel, which facilitates the drug release at occlusion, as well as the sizing and placement of medical devices like stents and balloons for following treatment.

When there is a severe occlusion with only tiny microchannels for the blood to flow through, the tool can utilize the force from the blood to easily slide through these narrow channels. Which way was chosen indicates to the surgeon which access route to take for the following medical operation.

“The methods of diagnosing and treating endovascular narrow diseases such as vascular stenosis or chronic total occlusion are still very limited. It is difficult to accurately detect and cross these areas in the very complex network of vessels inside the body”, says Yingbo Yan, who is a guest researcher in the Physical Intelligence Department at MPI-IS. He is the first author of the paper “Magnetically-assisted soft milli-tools for occluded lumen morphology detection”, which was published in Science Advances on August 18, 2023. “We hope that our new soft robotic tool can one day help accurately detect and navigate through the many complex and narrow vessels inside a body, and perform treatments more effectively, reducing potential risks.”

This tiny and soft tool has a 20 mm long magnetic Active Deformation Segment (ADS) and a 5mm long Fluid Drag-driven Segment (FDS). The magnetization profile of ADS is pre-programmed with a vibrating-sample magnetometer, providing a uniform magnetic field. Under an external magnetic field, this part can deform into a sinusoidal shape, easily adapting to the surrounding environment and deforming into various shapes. Thus, continuous monitoring of the shape changes of ADS while retracting it can provide detailed morphological information of the partial occlusions inside a vessel.

The FDS was fabricated using a soft polymer. Small beams on its side are bent by the fluidic drag from the incoming flow. In this way, the entire tool is carried towards the area with the highest flow velocity. Therefore, learning the location of the FDS while advancing it can point to the location and the route of the microchannel inside the severe occlusions.

“Detection of vascular diseases in the distal and hard-to-reach vascular regions such as the brain can be more challenging clinically, and our tool could work with Stentbot in the untethered mode”, says Tianlu Wang, a postdoc in the Physical Intelligence Department at MPI-IS and another first author of the work. “Stentbot is a wireless robot used for locomotion and medical functions in the distal vasculature we recently developed in our research group. We believe this new soft robotic tool can add new capabilities to wireless robots and contribute new solutions in these challenging regions.”

“Our tool shows potential to greatly improve minimally invasive medicine. This technology can reach and detect areas that were previously difficult to access. We expect that our robot can help make the diagnosis and treatment of, for instance, stenosis or a CTO more precise and timelier”, says Metin Sitti, Director of the Physical Intelligence Department at MPI-IS, Professor at Koç University and ETH Zurich.

• PAPER – Magnetically assisted soft milli-tools for occluded lumen morphology detection. Yingbo Yan, Tianlu Wang, Rongjing Zhang, Yilun Liu, Wenqi Hu and Metin Sitti. Science Advances, 9(33), eadi3979.

View Details

By Matt Shipman

Researchers who created a soft robot that could navigate simple mazes without human or computer direction have now built on that work, creating a “brainless” soft robot that can navigate more complex and dynamic environments.

“In our earlier work, we demonstrated that our soft robot was able to twist and turn its way through a very simple obstacle course,” says Jie Yin, co-corresponding author of a paper on the work and an associate professor of mechanical and aerospace engineering at North Carolina State University. “However, it was unable to turn unless it encountered an obstacle. In practical terms this meant that the robot could sometimes get stuck, bouncing back and forth between parallel obstacles.

“We’ve developed a new soft robot that is capable of turning on its own, allowing it to make its way through twisty mazes, even negotiating its way around moving obstacles. And it’s all done using physical intelligence, rather than being guided by a computer.”

Physical intelligence refers to dynamic objects – like soft robots – whose behavior is governed by their structural design and the materials they are made of, rather than being directed by a computer or human intervention.

As with the earlier version, the new soft robots are made of ribbon-like liquid crystal elastomers. When the robots are placed on a surface that is at least 55 degrees Celsius (131 degrees Fahrenheit), which is hotter than the ambient air, the portion of the ribbon touching the surface contracts, while the portion of the ribbon exposed to the air does not. This induces a rolling motion; the warmer the surface, the faster the robot rolls.

However, while the previous version of the soft robot had a symmetrical design, the new robot has two distinct halves. One half of the robot is shaped like a twisted ribbon that extends in a straight line, while the other half is shaped like a more tightly twisted ribbon that also twists around itself like a spiral staircase.

This asymmetrical design means that one end of the robot exerts more force on the ground than the other end. Think of a plastic cup that has a mouth wider than its base. If you roll it across the table, it doesn’t roll in a straight line – it makes an arc as it travels across the table. That’s due to its asymmetrical shape.

“The concept behind our new robot is fairly simple: because of its asymmetrical design, it turns without having to come into contact with an object,” says Yao Zhao, first author of the paper and a postdoctoral researcher at NC State. “So, while it still changes directions when it does come into contact with an object – allowing it to navigate mazes – it cannot get stuck between parallel objects. Instead, its ability to move in arcs allows it to essentially wiggle its way free.”

The researchers demonstrated the ability of the asymmetrical soft robot design to navigate more complex mazes – including mazes with moving walls – and fit through spaces narrower than its body size. The researchers tested the new robot design on both a metal surface and in sand.

“This work is another step forward in helping us develop innovative approaches to soft robot design – particularly for applications where soft robots would be able to harvest heat energy from their environment,” Yin says.

The paper, “Physically Intelligent Autonomous Soft Robotic Maze Escaper,” appears in the journal Science Advances. First author of the paper is Yao Zhao, a postdoctoral researcher at NC State. Hao Su, an associate professor of mechanical and aerospace engineering at NC State, is co-corresponding author. Additional co-authors include Yaoye Hong, a recent Ph.D. graduate of NC State; Yanbin Li, a postdoctoral researcher at NC State; and Fangjie Qi and Haitao Qing, both Ph.D. students at NC State.

The work was done with support from the National Science Foundation under grants 2005374, 2126072, 1944655 and 2026622.

• PAPER – Physically Intelligent Autonomous Soft Robotic Maze Escaper. Yao Zhao, Yaoye Hong, Yanbin Li, Fangjie Qi, Haitao Qing and Jie Yin. Science Advances, 9(36), eadi3254.

View Details

Researchers at the University of Washington developed small robotic devices that can change how they move through the air by “snapping” into a folded position during their descent. Shown here is a timelapse photo of the “microflier” falling in its unfolded state, which makes it tumble chaotically and spread outward in the wind. Photo by Mark Stone/University of Washington

By Roger Van Scyoc

On a cool afternoon at the heart of the University of Washington’s campus, autumn, for a few fleeting moments, appears to have arrived early. Tiny golden squares resembling leaves flutter then fall, switching from a frenzied tumble to a graceful descent with a snap.

Aptly named “microfliers” and inspired by Miura-fold origami, these small robotic devices can fold closed during their descent after being dropped from a drone. This “snapping” action changes the way they disperse and may, in the future, help change the way scientists study agriculture, meteorology, climate change and more.

“In nature, you see leaves and seeds disperse in just one manner,” said Kyle Johnson, an Allen School Ph.D. student and a first co-author of the paper on the subject published in Science Robotics. “What we were able to achieve was a structure that can actually act in two different ways.”

When open flat, the devices tumble chaotically, mimicking the descent of an elm leaf. When folded closed, they drop in a more stable manner, mirroring how a maple leaf falls from a branch. Through a number of methods — onboard pressure sensor, timer or a Bluetooth signal — the researchers can control when the devices transition from open to closed, and in doing so, manipulate how far they disperse through the air.

How could they achieve this? By reading between the lines.

“The Miura-ori origami fold, inspired by geometric patterns found in leaves, enables the creation of structures that can ‘snap’ between a flat and more folded state,” said co-senior author Vikram Iyer, an Allen School professor and co-director of the Computing for the Environment (CS4Env) initiative. “Because it only takes energy to switch between the states, we began exploring this as an energy efficient way to change surface area in mid-air, with the intuition that opening or closing a parachute will change how fast an object falls.”

That energy efficiency is key to being able to operate without batteries and scale down the fliers’ size and weight. Fitted with a battery-free actuator and a solar power-harvesting circuit, microfliers boast energy-saving features not seen in larger and heavier battery-powered counterparts such as drones. Yet they are robust enough to carry sensors for a number of metrics, including temperature, pressure, humidity and altitude. Beyond measuring atmospheric conditions, the researchers say a network of these devices could help paint a picture of crop growth on farmland or detect gas leaks near population centers.

“This approach opens up a new design space for microfliers by using origami,” said Shyam Gollakota, the Thomas J. Cable Endowed Professor in the Allen School and director of the school’s Mobile Intelligence Lab who was also a co-senior author. “We hope this work is the first step towards a future vision for creating a new class of fliers and flight modalities.”

Weighing less than half a gram, microfliers require less material and cost less than drones. They also offer the ability to go where it’s too dangerous for a human to set foot.

For instance, Johnson said, microfliers could be deployed when tracking forest fires. Currently, firefighting teams sometimes rappel down to where a fire is spreading. Microfliers could assist in mapping where a fire may be heading and where best to drop a payload of water. Furthermore, the team is working on making more components of the device biodegradable in the case that they can’t be recovered after being released.

“There’s a good amount of work toward making these circuits more sustainable,” said Vicente Arroyos, another Allen School Ph.D. student and first co-author on the paper. “We can leverage our work on biodegradable materials to make these more sustainable.”

Besides improving sustainability, the researchers also tackled challenges relating to the structure of the device itself. Early prototypes lacked the carbon fiber roots that provide the rigidity needed to prevent accidental transitions between states.

The research team took inspiration from elm and maple leaves in designing the microfliers. When open flat, the devices tumble chaotically, similar to how an elm leaf falls from a branch. When they are “snapped” into a folded position, as shown here, they descend in a more stable, straight downward manner like a maple leaf. Photo by Mark Stone/University of Washington

Collecting maple and elm leaves from outside their lab, the researchers noticed that while their origami structures exhibited the bistability required to change between states, they flexed too easily and didn’t have the venation seen in the found foliage. To gain more fine-grained control, they took another cue from the environment.

“We looked again to nature to make the faces of the origami flat and rigid, adding a vein-like pattern to the structure using carbon fiber,” Johnson said. “After that modification, we no longer saw a lot of the energy that we input dissipate over the origami’s faces.”

In total, the researchers estimate that the development of their design took about two years. There’s still room to grow, they added, noting that the current microfliers can only transition from open to closed. They said newer designs, by offering the ability to switch back and forth between states, may offer more precision and flexibility in where and how they’re used.

During testing, when dropped from an altitude of 40 meters, for instance, the microfliers could disperse up to distances of 98 meters in a light breeze. Further refinements could increase the area of coverage, allowing them to follow more precise trajectories by accounting for variables such as wind and inclement conditions.

Related to their previous work with dandelion-inspired sensors, the origami microfliers build upon the researchers’ larger goal of creating the internet of bio-inspired things. Whereas the dandelion-inspired devices featured passive flight, reflecting the manner in which dandelion seeds disperse through the wind, the origami microfliers function as complete robotic systems that include actuation to change their shape, active and bi-directional wireless transmission via an onboard radio, and onboard computing and sensing to autonomously trigger shape changes upon reaching a target altitude.

“This design can also accommodate additional sensors and payload due to its size and power harvesting capabilities,” Arroyos said. “It’s exciting to think about the untapped potential for these devices.”

The future, in other words, is quickly taking shape.

“Origami is inspired by nature,” Johnson added, smiling. “These patterns are all around us. We just have to look in the right place.”

The project was an interdisciplinary work by an all-UW team. The paper’s co-authors also included Amélie Ferran, a Ph.D. student in the mechanical engineering department, as well as Raul Villanueva, Dennis Yin and Tilboon Elberier, who contributed as undergraduate students studying electrical and computer engineering, and mechanical engineering professors Alberto Aliseda and Sawyer Fuller.

Johnson and Arroyos, who co-founded and currently lead the educational nonprofit AVELA – A Vision for Engineering Literacy & Access, and their teammates have done outreach efforts in Washington state K-12 schools related to the research, including showing students how to create their own bi-stable leaf-out origami structure using a piece of paper. Check out a related demonstration video here, and learn more about the microflier project here and in a related UW News release and GeekWire story.

• PAPER – Solar-powered shape-changing origami microfliers. Kyle Johnson, Vicente Arroyos, Amélie Ferran, Raul Villanueva, Dennis Yin, Alberto Aliseda, Sawyer Fuller, Vikram Iyer, and Shyamnath Gollakota. Science Robotics 8.82 (2023): eadg4276.

View Details

Virtual-reality technology could help cure people of phobias including about spiders. © Leena Robinson, Shutterstock.com

By Helen Massy-Beresford

Imagine a single technology that could help a robot perform safety checks at a nuclear plant, cure a person’s arachnophobia and simulate the feeling of a hug from a distant relative.

Welcome to the world of “extended reality”. Researchers funded by the EU have sought to demonstrate its enormous potential.

Relevant researchTheir goal was to make augmented reality, in which the real world is digitally enhanced, and virtual reality – a fully computer-generated environment – more immersive for users.

One of the researchers, Erik Hernandez Jimenez, never imagined the immediate relevance of a project that he led when it started in mid-2019. Within a year, the Covid-19 pandemic had triggered countless lockdowns that left people working and socialising through video connections from home.

‘We thought about how to apply this technology, how to feel human touch even at a distance, when we were all locked at home and contact with others was through a computer,’ said Hernandez Jimenez.

He coordinated the EU research initiative, which was named TACTILITY and ran from July 2019 until the end of September 2022.

The TACTILITY team developed a glove that simulates the sense of touch. Users have the sensation of touching virtual objects through electrical pulses delivered by electrodes embedded in the glove.

The sensations range from pushing a button and feeling pressure on the finger to handling a solid object and feeling its shape, dimensions and texture.

Glove and suit‘TACTILITY is about including tactile feedback in a virtual-reality scenario,’ said Hernandez Jimenez, who is a project manager at Spanish research institute TECNALIA.

He said the principle could be extended from the glove to a whole body suit.

Compared with past attempts to simulate touch sensations with motors, the electro-tactile feedback technique produces a more realistic result at a lower cost, according to Hernandez Jimenez.

This opens up the possibility of making the technology more widely accessible.

The research bolsters European Commission efforts to develop the virtual-worlds domain, which could provide 860 000 new jobs in Europe this decade as the worldwide sector grows from €27 billion in 2022.

The EU has around 3 700 companies, research organisations and governmental bodies that operate in this sphere, according to the Commission.

Phobias to factoriesThe TACTILITY researchers looked at potential healthcare applications.

“We thought about how to apply this technology, how to feel human touch even at a distance.”

– Erik Hernandez Jimenez, TACTILITY

That’s where spiders come into the picture. They were among the objects in the project’s experiments to mimic touch.

‘One that was quite impressive – although I didn’t like it at all – was feeling a spider or a cockroach crawling over your hand,’ Hernandez Jimenez said.

A potential use for the technology is treating phobias through exposure therapy in which patients are gradually desensitised to the source of their fear. That could start by virtually “touching” cartoon-like creepy crawlies before progressing to more lifelike versions.

The tactile glove can also be used in the manufacturing industry, helping the likes of car manufacturers train their workers to perform tricky manoeuvres on the factory floor.

Furthermore, it can help people collaborate more effectively with remotely controlled robots in hazardous environments. An example is a nuclear power plant, where a person in a control room can virtually “feel” what a robot is touching.

‘They get another sense and another kind of feedback, with more information to perform better checks,’ Hernandez Jimenez said.

Joyful and playfulWearables for virtual reality. © Oğuz ‘Oz’ Buruk, 2021

Wearable technologies for virtual-reality environments are also being inspired by the gaming industry.

Researchers in a second EU-funded project sought to expand the prospects for technologies already widely used for professional purposes. The initiative, called WEARTUAL, ran from May 2019 until late 2021.

“Wearables are fashion items – they’re part of the way we construct our identity.”

– Oğuz ‘Oz’ Buruk, WEARTUAL

‘Our project focused on the more experiential side – joyful and playful activities,’ said Oğuz ‘Oz’ Buruk, who coordinated WEARTUAL and is assistant professor of gameful experience at Tampere University in Finland.

Until recently, experiencing a virtual-reality environment involved a hand-held controller or head-mounted display.

The WEARTUAL researchers looked at ways of incorporating wearables worn, for example, on the wrist or ankle into virtual reality to give people a sense of greater immersion.

That could mean having their avatar – a representative icon or figure in the virtual world – blush when nervous or excited to enhance their ability to express themselves.

On the cuspThe team developed a prototype that could integrate varying physical sensations into the virtual world by transferring to it real-life data such as heart rate.

Buruk is interested in how games will look in the “posthuman” era, when people and machines increasingly converge through bodily implants, robotics and direct communication between the human brain and computers.

He signals that it’s hard to overestimate the eventual impact of advances in this area on everyday life, albeit over varying timescales: wearables are likely to be much more widely used in virtual reality in the next decade, while widespread use of bodily implants is more likely to take 50 to 100 years.

As technology and human bodies become ever more closely linked, the experience of transferring them to a virtual world will be enhanced, encouraging people to spend increasing amounts of time there, according to Buruk.

Virtual-reality technologies are already being used for practical purposes such as gamifying vital information including fire-safety procedures, making it more interactive and easier to learn. This type of use could expand to many areas.

On a very different front, several fashion houses already sell clothes that can be worn in virtual environments, allowing people to express their identity and creativity.

‘Wearables are fashion items – they’re part of the way we construct our identity,’ Buruk said. ‘Investments in virtual reality, extended reality and augmented reality are increasing every day.’

Research in this article was funded by the EU via the Marie Skłodowska-Curie Actions (MSCA).

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

By Deborah Pirchner

Malaria is an infectious disease claiming more than half a million lives each year. Because traditional diagnosis takes expertise and the workload is high, an international team of researchers investigated if diagnosis using a new system combining an automatic scanning microscope and AI is feasible in clinical settings. They found that the system identified malaria parasites almost as accurately as experts staffing microscopes used in standard diagnostic procedures. This may help reduce the burden on microscopists and increase the feasible patient load.

Each year, more than 200 million people fall sick with malaria and more than half a million of these infections lead to death. The World Health Organization recommends parasite-based diagnosis before starting treatment for the disease caused by Plasmodium parasites. There are various diagnostic methods, including conventional light microscopy, rapid diagnostic tests and PCR.

The standard for malaria diagnosis, however, remains manual light microscopy, during which a specialist examines blood films with a microscope to confirm the presence of malaria parasites. Yet, the accuracy of the results depends critically on the skills of the microscopist and can be hampered by fatigue caused by excessive workloads of the professionals doing the testing.

Now, writing in Frontiers in Malaria, an international team of researchers has assessed whether a fully automated system, combining AI detection software and an automated microscope, can diagnose malaria with clinically useful accuracy.

“At an 88% diagnostic accuracy rate relative to microscopists, the AI system identified malaria parasites almost, though not quite, as well as experts,” said Dr Roxanne Rees-Channer, a researcher at The Hospital for Tropical Diseases at UCLH in the UK, where the study was performed. “This level of performance in a clinical setting is a major achievement for AI algorithms targeting malaria. It indicates that the system can indeed be a clinically useful tool for malaria diagnosis in appropriate settings.”

AI delivers accurate diagnosisThe researchers sampled more than 1,200 blood samples of travelers who had returned to the UK from malaria-endemic countries. The study tested the accuracy of the AI and automated microscope system in a true clinical setting under ideal conditions.

They evaluated samples using both manual light microscopy and the AI-microscope system. By hand, 113 samples were diagnosed as malaria parasite positive, whereas the AI-system correctly identified 99 samples as positive, which corresponds to an 88% accuracy rate.

“AI for medicine often posts rosy preliminary results on internal datasets, but then falls flat in real clinical settings. This study independently assessed whether the AI system could succeed in a true clinical use case,” said Rees-Channer, who is also the lead author of the study.

Automated vs manualThe fully automated malaria diagnostic system the researchers put to the test includes hard- as well as software. An automated microscopy platform scans blood films and malaria detection algorithms process the image to detect parasites and the quantity present.

Automated malaria diagnosis has several potential benefits, the scientists pointed out. “Even expert microscopists can become fatigued and make mistakes, especially under a heavy workload,” Rees-Channer explained. “Automated diagnosis of malaria using AI could reduce this burden for microscopists and thus increase the feasible patient load.” Furthermore, these systems deliver reproducible results and can be widely deployed, the scientists wrote.

Despite the 88% accuracy rate, the automated system also falsely identified 122 samples as positive, which can lead to patients receiving unnecessary anti-malarial drugs. “The AI software is still not as accurate as an expert microscopist. This study represents a promising datapoint rather than a decisive proof of fitness,” Rees-Channer concluded.

Read the research in fullEvaluation of an automated microscope using machine learning for the detection of malaria in travelers returned to the UK, Roxanne R. Rees-Channer, Christine M. Bachman, Lynn Grignard, Michelle L. Gatton, Stephen Burkot, Matthew P. Horning, Charles B. Delahunt, Liming Hu, Courosh Mehanian, Clay M. Thompson, Katherine Woods, Paul Lansdell, Sonal Shah, Peter L. Chiodini, Frontiers in Malaria (2023).

View Details

In the realm of disaster response, technology plays a pivotal role in aiding communities during challenging times. In this exploration, we turn our attention to drones and their application in earthquake response, especially as how they are being used in the recent Morocco earthquake. This concise video offers valuable insights into the practical uses of […]

View Details

Sharifa Alghowinem, a research scientist in the Media Lab’s Personal Robots Group, poses with Jibo, a friendly robot companion developed by Professor Cynthia Breazeal. Credits: Gretchen Ertl

By Dorothy Hanna | Department of Mechanical Engineering

“As a child, I wished for a robot that would explain others’ emotions to me” says Sharifa Alghowinem, a research scientist in the Media Lab’s Personal Robots Group (PRG). Growing up in Saudi Arabia, Alghowinem says she dreamed of coming to MIT one day to develop Arabic-based technologies, and of creating a robot that could help herself and others navigate a complex world.

In her early life, Alghowinem faced difficulties with understanding social cues and never scored well on standardized tests, but her dreams carried her through. She earned an undergraduate degree in computing before leaving home to pursue graduate education in Australia. At the Australian National University, she discovered affective computing for the first time and began working to help AI detect human emotions and moods, but it wasn’t until she came to MIT as a postdoc with the Ibn Khaldun Fellowship for Saudi Arabian Women, which is housed in the MIT Department of Mechanical Engineering, that she was finally able to work on a technology with the potential to explain others’ emotions in English and Arabic. Today, she says her work is so fun that she calls the lab “my playground.”

Alghowinem can’t say no to an exciting project. She found one with great potential to make robots more helpful to people by working with Jibo, a friendly robot companion developed by the founder of the Personal Robots Group (PRG) and the social robot startup Jibo Inc., MIT Professor and Dean for Digital Learning Cynthia Breazeal. Breazeal’s research explores the potential for companion robots to go far beyond assistants who obey transactional commands, like requests for the daily weather, adding items to shopping lists, or controlling lighting. At the MIT Media Lab, the PRG team designs Jibo to make him an insightful coach and companion to advance social robotics technologies and research. Visitors to the MIT Museum can experience Jibo’s charming personality.

Alghowinem’s research has focused on mental health care and education, often working with other graduate students and Undergraduate Research Opportunity Program students in the group. In one study, Jibo coached young and older adults via positive psychology. He adapted his interventions based on the verbal and non-verbal responses he observed in the participants. For example, Jibo takes in the verbal content of a participant’s speech and combines it with non-verbal information like prolonged pauses and self-hugs. If he concludes that deep emotions have been disclosed, Jibo responds with empathy. When the participant doesn’t disclose, Jibo asks a gentle follow up question like, “Can you tell me more?”

Another project studied how a robot can effectively support high-quality parent and child interactions while reading a storybook together. Multiple PRG studies work together to learn what types of data are needed for a robot to understand people’s social and emotional states.

Research Scientist Sharifa Alghowinem (left) and visiting students Deim Alfozan and Tasneem Burghleh from Saudi Arabia’s Prince Sultan University, interact with Jibo. Credits: Gretchen Ertl

“I would like to see Jibo become a companion for the whole household,” says Alghowinem. Jibo can take on different roles with different family members such as a companion, reminding elders to take medication, or as a playmate for children. Alghowinem is especially motivated by the unique role Jibo could play in emotional wellness, and playing a preventative role in depression or even suicide. Integrating Jibo into daily life provides the opportunity for Jibo to detect emerging concerns and intervene, acting as a confidential resource or mental health coach.

Alghowinem is also passionate about teaching and mentoring others, and not only via robots. She makes sure to meet individually with the students she mentors every week and she was instrumental earlier this year in bringing two visiting undergraduate students from Prince Sultan University in Saudi Arabia. Mindful of their social-emotional experience, she worked hard to create the opportunity for the two students, together, to visit MIT so they could support each other. One of the visiting students, Tasneem Burghleh, says she was curious to meet the person who went out of her way to make opportunities for strangers and discovered in her an “endless passion that makes her want to pass it on and share it with everyone else.”

Next, Alghowinem is working to create opportunities for children who are refugees from Syria. Still in the fundraising stage, the plan is to equip social robots to teach the children English language and social-emotional skills and provide activities to preserve cultural heritage and Arabic abilities.

“We’ve laid the groundwork by making sure Jibo can speak Arabic as well as several other languages,” says Alghowinem. “Now I hope we can learn how to make Jibo really useful to kids like me who need some support as they learn how to interact with the world around them.”

View Details

This illustration shows a 3D printed heart ventricle engineered with fiber-infused ink. Credit: Harvard SEAS

By Kat J. McAlpine / SEAS Communications

Over the last decade, advances in 3D printing have unlocked new possibilities for bioengineers to build heart tissues and structures. Their goals include creating better in vitro platforms for discovering new therapeutics for heart disease, the leading cause of death in the United States, responsible for about one in every five deaths nationally, and using 3D-printed cardiac tissues to evaluate which treatments might work best in individual patients. A more distant aim is to fabricate implantable tissues that can heal or replace faulty or diseased structures inside a patient’s heart.

In a paper published in Nature Materials, researchers from Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) and the Wyss Institute for Biologically Inspired Engineering at Harvard University report the development of a new hydrogel ink infused with gelatin fibers that enables 3D printing of a functional heart ventricle that mimics beating like a human heart. They discovered the fiber-infused gel (FIG) ink allows heart muscle cells printed in the shape of a ventricle to align and beat in coordination like a human heart chamber.

“People have been trying to replicate organ structures and functions to test drug safety and efficacy as a way of predicting what might happen in the clinical setting,” says Suji Choi, research associate at SEAS and first author on the paper. But until now, 3D printing techniques alone have not been able to achieve physiologically-relevant alignment of cardiomyocytes, the cells responsible for transmitting electrical signals in a coordinated fashion to contract heart muscle.

“We started this project to address some of the inadequacies in 3D printing of biological tissues.”

– Kevin “Kit” Parker

The innovation lies in the addition of fibers within a printable ink. “FIG ink is capable of flowing through the printing nozzle but, once the structure is printed, it maintains its 3D shape,” says Choi. “Because of those properties, I found it’s possible to print a ventricle-like structure and other complex 3D shapes without using extra support materials or scaffolds.”

This video shows the spontaneous beating of a 3D-printed heart muscle. Credit: Harvard SEAS.

To create the FIG ink, Choi leveraged a rotary jet spinning technique developed in the lab of Kevin “Kit” Parker, Ph.D. that fabricates microfiber materials using an approach similar to the way cotton candy is spun. Postdoctoral researcher and Wyss Lumineer Luke MacQueen, a co-author on the paper, proposed the idea that fibers created by the rotary jet spinning technique could be added to an ink and 3D printed. Parker is a Wyss Associate Faculty member and the Tarr Family Professor of Bioengineering and Applied Physics at SEAS.

“When Luke developed this concept, the vision was to broaden the range of spatial scales that could be printed with 3D printers by dropping the bottom out of the lower limits, taking it down to the nanometer scale,” Parker says. “The advantage of producing the fibers with rotary jet spinning rather than electrospinning” – a more conventional method for generating ultrathin fibers – “is that we can use proteins that would otherwise be degraded by the electrical fields in electrospinning.”

Using the rotary jet to spin gelatin fibers, Choi produced a sheet of material with a similar appearance to cotton. Next, she used sonification – sound waves – to break that sheet into fibers about 80 to 100 micrometers long and about 5 to 10 micrometers in diameter. Then, she dispersed those fibers into a hydrogel ink.

“This concept is broadly applicable – we can use our fiber-spinning technique to reliably produce fibers in the lengths and shapes we want.”

– Suji Choi

The most difficult aspect was troubleshooting the desired ratio between fibers and hydrogel in the ink to maintain fiber alignment and the overall integrity of the 3D-printed structure.

As Choi printed 2D and 3D structures using FIG ink, the cardiomyocytes lined up in tandem with the direction of the fibers inside the ink. By controlling the printing direction, Choi could therefore control how the heart muscle cells would align.

The tissue-engineered 3D ventricle model. Credit: Harvard SEAS

When she applied electrical stimulation to 3D-printed structures made with FIG ink, she found it triggered a coordinated wave of contractions in alignment with the direction of those fibers. In a ventricle-shaped structure, “it was very exciting to see the chamber actually pumping in a similar way to how real heart ventricles pump,” Choi says.

As she experimented with more printing directions and ink formulas, she found she could generate even stronger contractions within ventricle-like shapes.

“Compared to the real heart, our ventricle model is simplified and miniaturized,” she says. The team is now working toward building more life-like heart tissues with thicker muscle walls that can pump fluid more strongly. Despite not being as strong as real heart tissue, the 3D-printed ventricle could pump 5-20 times more fluid volume than previous 3D-printed heart chambers.

The team says the technique can also be used to build heart valves, dual-chambered miniature hearts, and more.

“FIGs are but one tool we have developed for additive manufacturing,” Parker says. “We have other methods in development as we continue our quest to build human tissues for regenerative therapeutics. The goal is not to be tool driven – we are tool agnostic in our search for a better way to build biology.”

Additional authors include Keel Yong Lee, Sean L. Kim, Huibin Chang, John F. Zimmerman, Qianru Jin, Michael M. Peters, Herdeline Ann M. Ardoña, Xujie Liu, Ann-Caroline Heiler, Rudy Gabardi, Collin Richardson, William T. Pu, and Andreas Bausch.

This work was sponsored by SEAS; the National Science Foundation through the Harvard University Materials Research Science and Engineering Center (DMR-1420570, DMR-2011754); the National Institutes of Health and National Center for Advancing Translational Sciences (UH3HL141798, 225 UG3TR003279); the Harvard University Center for Nanoscale Systems (CNS), a member of the National Nanotechnology Coordinated Infrastructure Network (NNCI) which is supported by the National Science Foundation (ECCS-2025158, S10OD023519); and the American Chemical Society’s Irving S. Sigal Postdoctoral Fellowships.

• PAPER – Fibre-infused gel scaffolds guide cardiomyocyte alignment in 3D-printed ventricles. Suji Choi, Keel Yong Lee, Sean L. Kim, Luke A. MacQueen, Huibin Chang, John F. Zimmerman, Qianru Jin, Michael M. Peters, Herdeline Ann M. Ardoña, Xujie Liu, Ann-Caroline Heiler, Rudy Gabardi, Collin Richardson, William T. Pu, Andreas R. Bausch and Kevin Kit Parker. Nat. Mater. 22, 1039–1046 (2023). https://doi.org/10.1038/s41563-023-01611-3

View Details

Picture from paper “Versatile multicontact planning and control for legged loco-manipulation“. © American Association for the Advancement of Science

We had the chance to interview Jean Pierre Sleiman, author of the paper “Versatile multicontact planning and control for legged loco-manipulation”, recently published in Science Robotics.

What is the topic of the research in your paper?
The research topic focuses on developing a model-based planning and control architecture that enables legged mobile manipulators to tackle diverse loco-manipulation problems (i.e., manipulation problems inherently involving a locomotion element). Our study specifically targeted tasks that would require multiple contact interactions to be solved, rather than pick-and-place applications. To ensure our approach is not limited to simulation environments, we applied it to solve real-world tasks with a legged system consisting of the quadrupedal platform ANYmal equipped with DynaArm, a custom-built 6-DoF robotic arm.

Could you tell us about the implications of your research and why it is an interesting area for study?
The research was driven by the desire to make such robots, namely legged mobile manipulators, capable of solving a variety of real-world tasks, such as traversing doors, opening/closing dishwashers, manipulating valves in an industrial setting, and so forth. A standard approach would have been to tackle each task individually and independently by dedicating a substantial amount of engineering effort to handcraft the desired behaviors:

This is typically achieved through the use of hard-coded state-machines in which the designer specifies a sequence of sub-goals (e.g., grasp the door handle, open the door to a desired angle, hold the door with one of the feet, move the arm to the other side of the door, pass through the door while closing it, etc.). Alternatively, a human expert may demonstrate how to solve the task by teleoperating the robot, recording its motion, and having the robot learn to mimic the recorded behavior.

However, this process is very slow, tedious, and prone to engineering design errors. To avoid this burden for every new task, the research opted for a more structured approach in the form of a single planner that can automatically discover the necessary behaviors for a wide range of loco-manipulation tasks, without requiring any detailed guidance for any of them.

Could you explain your methodology?
The key insight underlying our methodology was that all of the loco-manipulation tasks that we aimed to solve can be modeled as Task and Motion Planning (TAMP) problems. TAMP is a well-established framework that has been primarily used to solve sequential manipulation problems where the robot already possesses a set of primitive skills (e.g., pick object, place object, move to object, throw object, etc.), but still has to properly integrate them to solve more complex long-horizon tasks.

This perspective enabled us to devise a single bi-level optimization formulation that can encompass all our tasks, and exploit domain-specific knowledge, rather than task-specific knowledge. By combining this with the well-established strengths of different planning techniques (trajectory optimization, informed graph search, and sampling-based planning), we were able to achieve an effective search strategy that solves the optimization problem.

The main technical novelty in our work lies in the Offline Multi-Contact Planning Module, depicted in Module B of Figure 1 in the paper. Its overall setup can be summarized as follows: Starting from a user-defined set of robot end-effectors (e.g., front left foot, front right foot, gripper, etc.) and object affordances (these describe where the robot can interact with the object), a discrete state that captures the combination of all contact pairings is introduced. Given a start and goal state (e.g., the robot should end up behind the door), the multi-contact planner then solves a single-query problem by incrementally growing a tree via a bi-level search over feasible contact modes jointly with continuous robot-object trajectories. The resulting plan is enhanced with a single long-horizon trajectory optimization over the discovered contact sequence.

What were your main findings?
We found that our planning framework was able to rapidly discover complex multi- contact plans for diverse loco-manipulation tasks, despite having provided it with minimal guidance. For example, for the door-traversal scenario, we specify the door affordances (i.e., the handle, back surface, and front surface), and only provide a sparse objective by simply asking the robot to end up behind the door. Additionally, we found that the generated behaviors are physically consistent and can be reliably executed with a real legged mobile manipulator.

What further work are you planning in this area?
We see the presented framework as a stepping stone toward developing a fully autonomous loco-manipulation pipeline. However, we see some limitations that we aim to address in future work. These limitations are primarily connected to the task-execution phase, where tracking behaviors generated on the basis of pre-modeled environments is only viable under the assumption of a reasonably accurate description, which is not always straightforward to define.

Robustness to modeling mismatches can be greatly improved by complementing our planner with data-driven techniques, such as deep reinforcement learning (DRL). So one interesting direction for future work would be to guide the training of a robust DRL policy using reliable expert demonstrations that can be rapidly generated by our loco-manipulation planner to solve a set of challenging tasks with minimal reward-engineering.

About the author

| | Jean-Pierre Sleiman received the B.E. degree in mechanical engineering from the American University of Beirut (AUB), Lebanon, in 2016, and the M.S. degree in automation and control from Politecnico Di Milano, Italy, in 2018. He is currently a Ph.D. candidate at the Robotic Systems Lab (RSL), ETH Zurich, Switzerland. His current research interests include optimization-based planning and control for legged mobile manipulation. |

• PAPER – Versatile multicontact planning and control for legged loco-manipulation. Jean-Pierre Sleiman, Farbod Farshidian and Marco Hutter. Science Robotics, 8(81), eadg5014.

View Details

As the last days of summer set, one is wistful of the time spent with loved ones sitting on the beach, traveling on the road, or just sharing a refreshing ice cream cone. However, for many Americans such emotional connections are rare, leading to high suicide rates and physical illness. In a recent study by the Surgeon General, more than half of the adults in the USA experience loneliness, with only 39% reporting feeling “very connected to others.” As Dr. Vivek H. Murthy states: “Loneliness is far more than just a bad feeling—it harms both individual and societal health. It is associated with a greater risk of cardiovascular disease, dementia, stroke, depression, anxiety, and premature death. The mortality impact of being socially disconnected is similar to that caused by smoking up to 15 cigarettes a day and even greater than that associated with obesity and physical inactivity.” In dollar terms, this epidemic accounts for close to $7 billion of Medicare spending annually, on top of $154 billion of yearly worker absenteeism.

As a Venture Capitalist, I have seen a growing number of pitch decks for conversational artificial intelligence in place of organic companions (some of these have wellness applications, while others are more lewd). One of the best illustrations of how AI-enabled chatbots are entering human relationships is in a recent article by the New York Times reporter Erin Griffin, who spent five days testing the AI buddy Pi. Near the end of the missive, she exclaims, “It wasn’t until Monday morning, after hours of intermittent chatting throughout the weekend, that I had my ‘aha’ moment with Pi. I was feeling overwhelmed with work and unsure of how to structure my day, a recurring hangup that often prevents me from getting started. ‘Good morning,’ I typed into the app. ‘I don’t have enough time to do everything I need to do today!’ With a level of enthusiasm only a robot could muster before coffee, Pi pushed me to break down my to-do list to create a realistic plan. Like much of the bot’s advice, it was obvious and simple, the kind of thing you would read in a self-help article by a productivity guru. But it was tailored specifically to me — and it worked.” As the reporter reflected on her weekend with the bot, she commented further, “I could have dumped my stress on a family member or texted a friend. But they are busy with their own lives and, well, they have heard this before. Pi, on the other hand, has infinite time and patience, plus a bottomless well of encouraging affirmations and detailed advice.”

In a population health study cited by General Murthy, the demographic that is most isolated in America is people over the age of 65. This is also the group that is most affected by physical and cognitive decline due to loneliness. Doctors Qi and Wu presented to Neurology Live a survey of the benefits of AI in their June paper, “ChatGPT: A Promising Tool to Combat Social Isolation and Loneliness in Older Adults With Mild Cognitive Impairment.” According to the authors, “ChatGPT can provide emotional support by offering a nonjudgmental space for individuals to express their thoughts and feelings. This can help alleviate loneliness and provide a sense of connection, which is crucial for well-being.” The researchers further cited ancillary uses, “ChatGPT can also assist with daily tasks and routines. By offering reminders for appointments, medications, and other daily tasks, this AI model can help older adults with MCI (mild cognitive impairment) maintain a sense of independence and control over their lives.” The problem with ChatGPT for geriatric plus populations is the form factors, as most seniors are not the most tech-savvy. This is an opportunity for roboticists.

Last Tuesday, Intuition Robotics announced it scored an additional financing of $25 million for expanding its “AI care companions” to all senior households. While its core product, ElliQ, does not move, its engagement offers the first glimpse of the benefits of social robots at mass. In speaking about the future, I interviewed its founder/CEO, Dor Skuler, last week. He shared with me his vision, “At this time, we don’t have plans to add legs or wheels to ElliQ, but we are always looking to add new activities or conversational features that can benefit the users. Our goal is to continue getting ElliQ into as many homes as possible to spread its benefits to even more older adults. We plan to create more partnerships with governments and aging agencies and are developing more partnerships within the healthcare industry. With this new funding, we will capitalize on our strong pipeline and fund the growth of our go-to-market activities.”

Unlike the stuffed animal executions of Paro and Tombot, ElliQ looks like an attractive home furnishing (and winner of the 2003 International Design Award). According to Skuler, this was very intentional, “We placed very high importance on the design of ElliQ to make it as easy as possible to use. We also knew we older adults needed technology that celebrated them and the aging process rather than focusing on disabilities and what they may no longer be able to do by themselves.” At the same time, the product underwent a rigorous testing and development stage that put its customer at the center of the process. “We designed ElliQ with the goal of helping seniors who are aging in place at home combat loneliness and social isolation. This group of seniors who participated in the development and beta testing helped us to shape and improve ElliQ, ensuring it had the right personality, character, mannerisms, and other modalities of interaction (like movement, conversation design, LEDs, and on-screen visuals) to form meaningful bonds with real people.” He further observed in the testing with hundreds of seniors, “we’ve witnessed older adults forming an actual relationship with ElliQ, closer to how one would see a roommate rather than a smart appliance.”

The results since deploying in homes throughout New York have been astounding in keeping older populations more socially and mentally engaged. As ElliQ’s creator elaborated, “In May 2022, we announced a partnership with the New York State Office for the Aging to bring 800+ ElliQ units to seniors across New York State at no cost to the end users. Just a few weeks ago this year, we announced a renewal of that partnership and the amazing results we’ve seen so far including a 95% reduction in loneliness and great improvement in well-being among older adults using the platform. ElliQ users throughout New York have demonstrated exceptionally high levels of engagement consistently over time, interacting with their ElliQ over 30 times per day, 6 days a week. More than 75% of these interactions are related to improving older adults’ social, physical, and mental well-being.”

To pedestrian cynics, ElliQ might look like an Alexa knockoff leading them to question why couldn’t the FAANG companies cannibalize the startup. Skuler’s response, “Alexa and other digital assistant technology were designed with the masses in mind or for younger end users. They also focus mainly on reactive AI, meaning they do not provide suggestions or talk with users unless prompted. ElliQ is designed to engage users over time, using a proactive approach to engagement. Its proactive suggestions and conversational capabilities foster a deep relationship with the user. Moreover, ElliQ’s integration of Generative AI and Large Language Models (LLMs) enables rich and continuous conversational experiences, allowing for more contextual, personalized, and goal-driven interactions. These capabilities and unique features such as drinking coffee with ElliQ in cafes around the world or visiting a virtual art museum, bring ElliQ and the user close together, creating trust that allows ElliQ to motivate the older adult to lead a more healthy and engaged lifestyle.”

While ElliQ and OpenAI’s ChatGPT have shown promise in treating mental illness, some health professionals are still not convinced. At MIT, professor and psychologist, Sherry Turkle, worries that the interactions of machines “push us along a road where we’re encouraged to forget what makes people special.” Dr. Turkle demures, “The performance of empathy is not empathy. The area of companion, lover therapist, best friend is really one of the few areas where people need people.”

View Details

Source: OpenAI’s DALL·E 2 with prompt “a hyperrealistic picture of a robot reading the news on a laptop at a coffee shop”

Welcome to the 4th edition of Robo-Insight, a biweekly robotics news update! In this post, we are excited to share a range of new advancements in the field and highlight robots’ progress in areas like mobile applications, cleaning, underwater mining, flexibility, human well-being, depression treatments, and human interactions.

Simplified mobile robot behavior adaptationsIn the world of system adaptions, researchers from Eindhoven University of Technology have introduced a methodology that bridges the gap between application developers and control engineers in the context of mobile robots’ behavior adaptation. This approach leverages symbolic descriptions of robots’ behavior, known as “behavior semantics,” and translates them into control actions through a “semantic map.” This innovation aims to simplify motion control programming for autonomous mobile robot applications and facilitate integration across various vendors’ control software. By establishing a structured interaction layer between application, interaction, and control layers, this methodology could streamline the complexity of mobile robot applications, potentially leading to more efficient underground exploration and navigation systems.

The frontal perspective of the mobile platform (showcases hardware components with blue arrows). Source.

New robot for household clean-upsSpeaking of helpful robots, Princeton University has created a robot named TidyBot to address the challenge of household tidying. Unlike simple tasks such as moving objects, real-world cleanup requires a robot to differentiate between objects, place them correctly, and avoid damaging them. TidyBot accomplishes this through a combination of physical dexterity, visual recognition, and language understanding. Equipped with a mobile robotic arm, a vision model, and a language model, TidyBot can identify objects, place them in designated locations, and even infer proper actions with an 85% accuracy rate. The success of TidyBot demonstrates its potential to handle complex household tasks.

TidyBot in work. Source.

Deep sea mining robotsShifting our focus to underwater environments, researchers are addressing the efficiency hurdles faced in deep-sea mining through innovative path planning for autonomous robotic mining vehicles. With deep-sea manganese nodules holding significant potential, these robotic vehicles are essential for their collection. By refining path planning methods, the researchers aim to improve the efficiency of these vehicles in traversing challenging underwater terrains while avoiding obstacles. This development could lead to more effective and responsible resource extraction from the ocean floor, contributing to the sustainable utilization of valuable mineral resources.

Diagram depicting the operational framework of the deep-sea mining system. Source.

Advanced soft robots with dexterity and flexibilityIn regards to the field of robotic motion, recently researchers from Shanghai Jiao Tong University have developed small-scale soft robots with remarkable dexterity, enabling immediate and reversible changes in motion direction and shape reconfiguration. These robots, powered by an active dielectric elastomer artificial muscle and a unique chiral-lattice foot design, can change direction during fast movement with a single voltage input. The chiral-lattice foot generates various locomotion behaviors, including forward, backward, and circular motion, by adjusting voltage frequencies. Additionally, combining this structural design with shape memory materials allows the robots to perform complex tasks like navigating narrow tunnels or forming specific trajectories. This innovation opens the door to next-generation autonomous soft robots capable of versatile locomotion.

The soft robot achieves circular motion in either right or left directions by positioning the lattice foot towards the respective sides. Source.

Robotic dogs utilized to comfort patientsTurning our focus to robot use in the healthcare field, Stanford students, along with researchers and doctors, have partnered with AI and robotics industry leaders to showcase new robotic dogs designed to interact with pediatric patients at Lucile Packard Children’s Hospital. Patients at the hospital had the opportunity to engage with the playful robots, demonstrating the potential benefits of these mechanical pets for children’s well-being during their hospital stays. The robots, called Pupper, were developed by undergraduate engineering students and operated using handheld controllers. The goal of the demonstration was to study the interaction between the robots and pediatric patients, exploring ways to enhance the clinical experience and reduce anxiety.

A patient playing with the robotic dog. Source.

Robotic innovations could help with depressionAlong the same lines as improving well-being, a recent pilot study has explored the potential benefits of using robotics in transcranial magnetic stimulation (TMS) for treating depression. Researchers led by Hyunsoo Shin developed a custom TMS robot designed to improve the accuracy of TMS coil placement on the brain, a critical aspect of effective treatment. By employing the robotic system, they reduced preparation time by 53% and significantly minimized errors in coil positioning. The study found comparable therapeutic effects on depression severity and regional cerebral blood flow (rCBF) between the robotic and manual TMS methods, shedding light on the potential of robotic assistance in enhancing the precision and efficiency of TMS treatments.

Configuration of the robotic repetitive transcranial magnetic stimulation (rTMS) within the treatment facility, and robotic positioning device for automated coil placement. Source.

Advanced robotic eye researchFinally, in the world of human-robot enhancement, a study conducted by researchers from various institutions has explored the potential of using robot eyes as predictive cues in human-robot interaction (HRI). The study aimed to understand whether and how the design of predictive robot eyes could enhance interactions between humans and robots. Four different types of eye designs were tested, including arrows, human eyes, and two anthropomorphic robot eye designs. The results indicated that abstract anthropomorphic robot eyes, which mimic certain aspects of human-like attention, were most effective at directing participants’ attention and triggering reflexive shifts. These findings suggest that incorporating abstract anthropomorphic eyes into robot design could improve the predictability of robot movements and enhance HRI.

The four types of stimuli. The first row showcases the human (left) and arrow (right) stimuli. The second row displays the abstract anthropomorphic robot eyes. Photograph of the questionnaire’s subject, the cooperative robot Sawyer. Source.

The continuous stream of progress seen across diverse domains underscores the adaptable and constantly progressing nature of robotics technology, revealing novel pathways for its incorporation across a spectrum of industries. The gradual advancement in the realm of robotics reflects persistent efforts and hints at the potential implications these strides might hold for the future.

Sources:

1. Chen, H. L., Hendrikx, B., Torta, E., Bruyninckx, H., & van de Molengraft, R. (2023, July 10). Behavior adaptation for mobile robots via semantic map compositions of constraint-based controllers. Frontiers.
2. Princeton Engineering – Engineers clean up with TidyBot. (n.d.). Princeton Engineering. Retrieved August 30, 2023,
3. Xie, Y., Liu, C., Chen, X., Liu, G., Leng, D., Pan, W., & Shao, S. (2023, July 12). Research on path planning of autonomous manganese nodule mining vehicle based on lifting mining system. Frontiers.
4. Wang, D., Zhao, B., Li, X., Dong, L., Zhang, M., Zou, J., & Gu, G. (2023). Dexterous electrical-driven soft robots with reconfigurable chiral-lattice foot design. Nature Communications, 14(1), 5067.
5. University, S. (2023, August 1). Robo-dogs unleash joy at Stanford hospital. Stanford Report.
6. Shin, H., Jeong, H., Ryu, W., Lee, G., Lee, J., Kim, D., Song, I.-U., Chung, Y.-A., & Lee, S. (2023). Robotic transcranial magnetic stimulation in the treatment of depression: a pilot study. Scientific Reports, 13(1), 14074.
7. Onnasch, L., Schweidler, P., & Schmidt, H. (2023, July 3). The potential of robot eyes as predictive cues in HRI-an eye-tracking study. Frontiers.

View Details

MIT researchers used kirigami, the art of Japanese paper cutting and folding, to develop ultrastrong, lightweight materials that have tunable mechanical properties, like stiffness and flexibility. These materials could be used in airplanes, automobiles, or spacecraft. Image: Courtesy of the researchers

By Adam Zewe | MIT News

Cellular solids are materials composed of many cells that have been packed together, such as a honeycomb. The shape of those cells largely determines the material’s mechanical properties, including its stiffness or strength. Bones, for instance, are filled with a natural material that enables them to be lightweight, but stiff and strong.

Inspired by bones and other cellular solids found in nature, humans have used the same concept to develop architected materials. By changing the geometry of the unit cells that make up these materials, researchers can customize the material’s mechanical, thermal, or acoustic properties. Architected materials are used in many applications, from shock-absorbing packing foam to heat-regulating radiators.

Using kirigami, the ancient Japanese art of folding and cutting paper, MIT researchers have now manufactured a type of high-performance architected material known as a plate lattice, on a much larger scale than scientists have previously been able to achieve by additive fabrication. This technique allows them to create these structures from metal or other materials with custom shapes and specifically tailored mechanical properties.

“This material is like steel cork. It is lighter than cork, but with high strength and high stiffness,” says Professor Neil Gershenfeld, who leads the Center for Bits and Atoms (CBA) at MIT and is senior author of a new paper on this approach.

The researchers developed a modular construction process in which many smaller components are formed, folded, and assembled into 3D shapes. Using this method, they fabricated ultralight and ultrastrong structures and robots that, under a specified load, can morph and hold their shape.

Because these structures are lightweight but strong, stiff, and relatively easy to mass-produce at larger scales, they could be especially useful in architectural, airplane, automotive, or aerospace components.

Joining Gershenfeld on the paper are co-lead authors Alfonso Parra Rubio, a research assistant in the CBA, and Klara Mundilova, an MIT electrical engineering and computer science graduate student; along with David Preiss, a graduate student in the CBA; and Erik D. Demaine, an MIT professor of computer science. The research will be presented at ASME’s Computers and Information in Engineering Conference.

The researchers actuate a corrugated structure by tensioning steel wires across the compliant surfaces and then connecting them to a system of pulleys and motors, enabling the structure to bend in either direction. Image: Courtesy of the researchers

Fabricating by foldingArchitected materials, like lattices, are often used as cores for a type of composite material known as a sandwich structure. To envision a sandwich structure, think of an airplane wing, where a series of intersecting, diagonal beams form a lattice core that is sandwiched between a top and bottom panel. This truss lattice has high stiffness and strength, yet is very lightweight.

Plate lattices are cellular structures made from three-dimensional intersections of plates, rather than beams. These high-performance structures are even stronger and stiffer than truss lattices, but their complex shape makes them challenging to fabricate using common techniques like 3D printing, especially for large-scale engineering applications.

The MIT researchers overcame these manufacturing challenges using kirigami, a technique for making 3D shapes by folding and cutting paper that traces its history to Japanese artists in the 7th century.

Kirigami has been used to produce plate lattices from partially folded zigzag creases. But to make a sandwich structure, one must attach flat plates to the top and bottom of this corrugated core onto the narrow points formed by the zigzag creases. This often requires strong adhesives or welding techniques that can make assembly slow, costly, and challenging to scale.

The MIT researchers modified a common origami crease pattern, known as a Miura-ori pattern, so the sharp points of the corrugated structure are transformed into facets. The facets, like those on a diamond, provide flat surfaces to which the plates can be attached more easily, with bolts or rivets.

The MIT researchers modified a common origami crease pattern, known as a Miura-ori pattern, so the sharp points of the corrugated structure are transformed into facets. The facets, like those on a diamond, provide flat surfaces to which the plates can be attached more easily, with bolts or rivets. Image: Courtesy of the researchers

“Plate lattices outperform beam lattices in strength and stiffness while maintaining the same weight and internal structure,” says Parra Rubio. “Reaching the H-S upper bound for theoretical stiffness and strength has been demonstrated through nanoscale production using two-photon lithography. Plate lattices construction has been so difficult that there has been little research on the macro scale. We think folding is a path to easier utilization of this type of plate structure made from metals.”

Customizable propertiesMoreover, the way the researchers design, fold, and cut the pattern enables them to tune certain mechanical properties, such as stiffness, strength, and flexural modulus (the tendency of a material to resist bending). They encode this information, as well as the 3D shape, into a creasing map that is used to create these kirigami corrugations.

For instance, based on the way the folds are designed, some cells can be shaped so they hold their shape when compressed while others can be modified so they bend. In this way, the researchers can precisely control how different areas of the structure will deform when compressed.

Because the flexibility of the structure can be controlled, these corrugations could be used in robots or other dynamic applications with parts that move, twist, and bend.

To craft larger structures like robots, the researchers introduced a modular assembly process. They mass produce smaller crease patterns and assemble them into ultralight and ultrastrong 3D structures. Smaller structures have fewer creases, which simplifies the manufacturing process.

Using the adapted Miura-ori pattern, the researchers create a crease pattern that will yield their desired shape and structural properties. Then they utilize a unique machine — a Zund cutting table — to score a flat, metal panel that they fold into the 3D shape.

“To make things like cars and airplanes, a huge investment goes into tooling. This manufacturing process is without tooling, like 3D printing. But unlike 3D printing, our process can set the limit for record material properties,” Gershenfeld says.

Using their method, they produced aluminum structures with a compression strength of more than 62 kilonewtons, but a weight of only 90 kilograms per square meter. (Cork weighs about 100 kilograms per square meter.) Their structures were so strong they could withstand three times as much force as a typical aluminum corrugation.

Using their method, researchers produced aluminum structures with a compression strength of more than 62 kilonewtons, but a weight of only 90 kilograms per square meter. Image: Courtesy of the researchers

The versatile technique could be used for many materials, such as steel and composites, making it well-suited for the production lightweight, shock-absorbing components for airplanes, automobiles, or spacecraft.

However, the researchers found that their method can be difficult to model. So, in the future, they plan to develop user-friendly CAD design tools for these kirigami plate lattice structures. In addition, they want to explore methods to reduce the computational costs of simulating a design that yields desired properties.

“Kirigami corrugations holds exciting potential for architectural construction,” says James Coleman MArch ’14, SM ’14, co-founder of the design for fabrication and installation firm SumPoint, and former vice president for innovation and R&D at Zahner, who was not involved with this work. “In my experience producing complex architectural projects, current methods for constructing large-scale curved and doubly curved elements are material intensive and wasteful, and thus deemed impractical for most projects. While the authors’ technology offers novel solutions to the aerospace and automotive industries, I believe their cell-based method can also significantly impact the built environment. The ability to fabricate various plate lattice geometries with specific properties could enable higher performing and more expressive buildings with less material. Goodbye heavy steel and concrete structures, hello lightweight lattices!”

Parra Rubio, Mundilova and other MIT graduate students also used this technique to create three large-scale, folded artworks from aluminum composite that are on display at the MIT Media Lab. Despite the fact that each artwork is several meters in length, the structures only took a few hours to fabricate.

“At the end of the day, the artistic piece is only possible because of the math and engineering contributions we are showing in our papers. But we don’t want to ignore the aesthetic power of our work,” Parra Rubio says.

This work was funded, in part, by the Center for Bits and Atoms Research Consortia, an AAUW International Fellowship, and a GWI Fay Weber Grant.

• PAPER – Kirigami corrugations: strong, modular, and programmable plate lattice. Alfonso Parra Rubio, Klara Mundilova, David Preiss, Erik D. Demaine, and Neil Gershenfeld.

View Details

An innovative bimanual robot displays tactile sensitivity close to human-level dexterity using AI to inform its actions.

View Details

With a new technique, a robot can reason efficiently about moving objects using more than just its fingertips.

View Details

In the last decade we have seen more robotics innovation becoming real products and companies than in the entire history of robotics. Furthermore, the greater Silicon Valley and San Francisco Bay Area is at the center of this ‘Cambrian Explosion in Robotics’ as Dr Gill Pratt, Director of Robotics at Toyota Research Institute described it. […]

View Details

Welcome to the third edition of Robo-Insight, a biweekly robotics news update! In this post, we are excited to share a range of new advancements in the field and highlight progress in areas like motion, unfamiliar navigation, dynamic control, digging, agriculture, surgery, and food sorting. A bioinspired robot masters 8 modes of motion for adaptive […]

View Details

A study found that adding legs does more for you than having a good sense of the ground around you − if you’re a mobile robot.

View Details

Even with the addition of a strange mineral, robots still obey the principle of bounded rationality in artificial intelligence set forth by Herb Simon. I cover bounded rationality in my Science Robotics review (image courtesy of @SciRobotics) but I am adding some more details here. Did you like the Western True Grit? Classic scifi like […]

View Details

Researchers at the Max Planck Institute for Intelligent Systems and the University of Colorado Boulder have developed a soft shape display, a robot that can rapidly and precisely change its surface geometry to interact with objects and liquids, react to human touch, and display letters and numbers – all at the same time. The display […]

View Details

By Angharad Brewer Gillham, Frontiers science writer Increasingly, social robots are being used for support in educational contexts. But does the sound of a social robot affect how well they perform, especially when dealing with teams of humans? Teamwork is a key factor in human creativity, boosting collaboration and new ideas. Danish scientists set out […]

View Details

2 years ago, I wrote A Guide to Docker and ROS, which is one of my most frequently viewed posts — likely because it is a tricky topic and people were seeking answers. Since then, I’ve had the chance to use Docker more in my work and have picked up some new tricks. This was […]

View Details

Roberto Figueiredo is a master’s student at the University of Aveiro. He is a member of the Bold Hearts RoboCup team which competes in the Humanoid KidSize soccer league. He is currently the local representative for the Junior Rescue Simulation. We spoke to Roberto about his RoboCup journey, from the junior to the major leagues, […]

View Details

By Helen Massy-Beresford Imagine seals swimming in the sea with electronic tags that send real-time water data to scientists back in their laboratories. Or archaeologists near a coast being automatically alerted when a diver trespasses on a precious shipwreck. Such scenarios are becoming possible as a result of underwater connected technologies, which can help monitor […]

View Details

By Aaron Aupperlee A research group in Carnegie Mellon University’s Robotics Institute is creating the next generation of explorers — robots. The Autonomous Exploration Research Team has developed a suite of robotic systems and planners enabling robots to explore more quickly, probe the darkest corners of unknown environments, and create more accurate and detailed maps. […]

View Details

Scientists at the Max Planck Institute for Intelligent Systems in Stuttgart have developed a magnetically controlled soft medical robot with a unique, flexible structure inspired by the body of a pangolin. The robot is freely movable despite built-in hard metal components. Thus, depending on the magnetic field, it can adapt its shape to be able […]

View Details

By Hayley Dunning and Caroline Brogan The prototype drone, called FireDrone, could be sent into burning buildings or woodland to assess hazards and provide crucial first-hand data from danger zones. The data would then be sent to first responders to help inform their emergency response. The drone is made of a new thermal aerogel insulation […]

View Details

Welcome to the 2nd edition of Robo-Insight, a biweekly robotics news update! In this post, we are excited to share a range of remarkable advancements in the field, showcasing progress in hazard mapping, surface crawling, pump controls, adaptive gripping, surgery, health assistance, and mineral extraction. These developments exemplify the continuous evolution and potential of robotics […]

View Details

In a new study, we demonstrate the potential of blockchain technology, known from cryptocurrencies such as Bitcoin and Ethereum, to secure the coordination of robot swarms. In experiments conducted with both real and simulated robots, we show how blockchain technology enables a robot swarm to neutralize harmful robots without human intervention, thus enabling the deployment […]

View Details

A new technique helps a nontechnical user understand why a robot failed, and then fine-tune it with minimal effort to perform a task effectively.

View Details

As this year’s RoboCup draws to a close, we take a look back at some of the highlights from the second half of the conference. Over the course of the weekend, the event focussed on the latter stages of the competitions, with the winners in all the different leagues decided. If you missed our round-up […]

View Details

This year’s RoboCup kicked off on 4 July and will run until 10 July. Taking place in Bordeaux, the event will see around 2500 participants, from 45 different countries take part in competitions, training sessions, and a symposium. Find out what attendees have been up to in preparation for, and in the first half of, […]

View Details

Last month, the entire world was abuzz when five über wealthy explorers perished at the bottom of the Atlantic Ocean near the grave of the once “unsinkable ship.” Disturbingly, during the same week, hundreds of war-torn refugees drowned in the Mediterranean with little news of their plight. The irony of machine versus nature illustrates how […]

View Details

New soft-bodied robots that can be controlled by a simple magnetic field are well suited to work in confined spaces.

View Details

Today’s robots are often static and isolated from humans in structured environments — you can think of robot arms employed by Amazon for picking and packaging products within warehouses. But the true potential of robotics lies in mobile robots operating alongside humans in messy environments like our homes and hospitals — this requires navigation skills. […]

View Details

Welcome to the inaugural edition of Robo-Insight, a biweekly robotics news update! In this post, we are thrilled to present a range of remarkable advancements in the field, highlighting robotics progress in terrain traversability, shape morphing, object avoidance, mechanical memory, physics-based AI techniques, and new home robotics kits. These developments exemplify the continuous evolution and […]

View Details

This year, RoboCup will be held in Bordeaux, from 4-10 July. The event will see around 2500 participants, from 45 different countries take part in competitions, training sessions, and a symposium. You can see the schedule for the week here. The leagues and their competitions The league competitions will take place on 6-9 July. You […]

View Details

By Inês Hipólito/Deborah Pirchner, Frontiers science writer Inês Hipólito is a highly accomplished researcher, recognized for her work in esteemed journals and contributions as a co-editor. She has received research awards including the prestigious Talent Grant from the University of Amsterdam in 2021. After her PhD, she held positions at the Berlin School of Mind […]

View Details

It is with great sadness that I am sharing that Joanne Pransky, the World’s First Robotic Psychariatrist, and who Isaac Asimov called the real Susan Calvin passed away recently. I had several delight conversations with her, including an interview and moderated panel. Joanne was a tireless advocate for robotics AND for women in robotics. She […]

View Details

Dramatic improvements in computing, sensors and submersible engineering are making it possible for researchers to ramp up data collection from the oceans while also keeping people out of harm’s way.

View Details

We are happy to dedicate this post to our new volunteer: Shaunak Kapur. Shaunak is a soon-to-be senior in high school (Texas), and he has been captivated by robotics from a young age. He has participated in numerous robotics competitions (namely VEX and FRC), pursued robotics/engineering internships and robotics-based research projects, and even worked to […]

View Details

In this special live recording of the Robot Talk podcast at the Great Exhibition Road Festival, Claire chatted to Glyn Morgan (Science Museum), Bani Anvari (University College London) and Thrishantha Nanayakara (Imperial College London) to explore how our intelligent friends from the world of science fiction match up with state-of-the art robotics and artificial intelligence […]

View Details

Artificial hands, even the most sophisticated prostheses, are still by far inferior to human hands. What they lack are the tactile abilities crucial for dexterity. Other challenges include linking sensing to action within the robotic system – and effectively linking it to the human user. Prof. Dr. Philipp Beckerle from FAU has joined with international […]

View Details

Finally, Intrinsic (a spin-off of Google-X) has revealed the product they have been working with the help of the Open Source Robotics Corporation team (among others): Flowstate! What is Flowstate? Flowstate is a web-based software designed to simplify ...

View Details

The same type of machine learning methods used to pilot self-driving cars and beat top chess players could help type-1 diabetes sufferers keep their blood glucose levels in a safe range.

View Details

Claire chatted to Robert Richardson from the University of Leeds all about 3D printing, robot design, and infrastructure repair. Robert Richardson is Professor of Robotics in the School of Mechanical Engineering at the University of Leeds, and executive chair of EPSRC UK-RAS network. His research interests include robotics for civil infrastructure inspection and repair, making […]

View Details

A new AI-based approach for controlling autonomous robots satisfies the often-conflicting goals of safety and stability.

View Details

If you missed the 2023 edition of the IEEE International Conference on Robotics and Automation in London, here we bring you the video digests that were made each of the main days of the conference. Enjoy! Tuesday 30th May Wednesday 31st May Thursday 1st June

View Details

Claire chatted to Sara Bernardini from Royal Holloway University of London all about decision-making, reconfigurable robots, and oceanography. Sara Bernardini is a Professor of AI at Royal Holloway University of London, the Principal Research Scientist in AI and Data Science at the National Oceanography Centre and a Fellow at the Alan Turing Institute. Her research […]

View Details

A simple sponge has improved how robots grasp, scientists from the University of Bristol have found.

View Details

In this post we bring you all the paper awards finalists and winners presented during the 2023 edition of the IEEE International Conference on Robotics and Automation (ICRA). Congratulations to the winners and finalists! ICRA 2023 Outstanding Paper Distributed Data-Driven Predictive Control for Multi-Agent Collaborative Legged Locomotion, by Fawcett, Randall; Amanzadeh, Leila; Kim, Jeeseop; Ames, […]

View Details

Is Rosie the Robot Maid from the Jetsons here yet? Several different types of humanoid are currently deployed commercially or in trials. We’ve come along way since the DARPA Robotics Challenge of 2015/2016, where the most popular footage was the blooper reels of robots falling over and failing to open doors or climb stairs.

The Avatar XPrize of 2019-2022 showcased some extremely sophisticated humanoids that certainly advanced the state of the art but the holy grail of humanoid robots is combining incredible sophistication into a sub $50,000 package. Why $50,000? Wouldn’t some companies pay a lot more? Then again, can’t we buy a car, also a very sophisticated device capable of partial autonomy that is 5 times the size of a humanoid, for less than $50,000? Why is this the benchmark for humanoids?

$50,000 is the annual wage for a single shift of labor at slightly more than $18/hour or minimum wage in every low wage industry. There is a terrible labor shortage and it is the dirty dull and dangerous jobs that are hardest for employers to fill. Companies that can afford to run two or more shifts a day also have more alternatives when it comes to filling their labor gaps. It’s the small to medium size enterprise that is suffering the most in our current economic and demographic conditions.

We don’t need a Six Million Dollar Man.

We need a $50,000 humanoid.

The roll out of sophisticated new robots and how we integrate them into society is at the heart of my early research and my current roles as the Managing Director of Silicon Valley Robotics (explain), VP of Global Robotics for AMT (explain) and the VP of Industrial Activities for the IEEE Robotics and Automation Society (explain).

As more and more companies announce their work towards the affordable humanoid robot, I wanted to create a reference chart for myself, and realized that it might be of interest to others as well. The ranking system is just my own opinion and it will be fascinating to see who succeeds and progresses over the next few years. Enjoy this overview and make up your own minds as to which humanoid robot is really the best.

Who’s in the running? (in alphabetical order by company not robot)* 1x – Eve * Aeolus Robotics – Eva * Agility Robotics – Digit * Apptronik – Astra * Boston Dynamics – Atlas * Comma.ai – body * Devanthro – Robody * Engineered Arts – Ameca * Figure – Figure01 * Giant.ai – Universal Worker * IIT – ErgoCub * PAL – Reem-C * Prosper Robotics – Alfie * Sanctuary – Phoenix * Tesla – Optimus * Toyota – T-HR3

Who isn’t in the running? Hollywood Humanoids
Hollywood Humanoids are one off robots for the purpose of entertainment, like Sophia from Hanson Robotics, Xoxe from AI Life, or Beonmi from Beyond Imagination. ….

Chinese robots
It’s too hard for me to validate that they exist, work as advertized, and what the specifications are.

Research robots
Love them but they have a different purpose. Only robots with commercial deployment plans, and ideally, a price tag and a date in 2023 or 2024 when they’ll be available for purchase, if they aren’t already being sold.

Not humanoid
I also love robots that work like a humanoid but don’t look human-like. We saw some examples in the DARPA Robotics Challenge, most notably RoboSimian. Once we go down that route, all quadrupeds, and multi-armed robots or wheeled humanlike robots, would qualify. Who knew there were so many robots!

Who have I missed?I’m hoping to crowdsource some more great robots :)

Read the original article on Substack.

View Details

Claire chatted to James Kell from Jacobs Engineering UK all about civil infrastructure, nuclear robotics and jet engine inspection.

James Kell is Robotics Technical Director at Jacobs Engineering UK. He is responsible for the internal robotics technical strategy and is actively looking to apply the decades worth of robotics and remote handling experience that Jacobs has to the wider market in other sectors. The overall intention is to extend the life of critical national infrastructure like roads, rail, and water. James’s previously worked for Rolls-Royce where his role was to develop keyhole surgery technologies to service jet engines. James is also a member of the Robotics Growth Partnership, supported by BEIS.

View Details

“Thirty million developers” are the answer to driving billion-dollar robot startups, exclaimed Eliot Horowitz of Viam last week at Automate. The hushed crowd of about 200 hardware entrepreneurs listened intensely to MongoDB‘s founder and former CTO (a $20Bn success story). Now, Horowitz aims to take the same approach that he took to democratizing cloud data applications to mechatronics. As I nudged him with questions about how his new platform will speed complex robot deployments to market, he shared his vision of the Viam developer army (currently 1,000+ strong) creating applications that can be seamlessly downloaded on the fly to any system and workflow. Unlike RoS which is primarily targeted to the current community of roboticists, Viam is luring the engineers that birthed ChatGPT to revolutionize uncrewed systems with new mechanical tasks addressing everyday needs. Imagine generative AI prompts for SLAM, gripping, computer vision, and other highly manipulative tasks with drag-and-drop ease.

Interviewing Horowitz recalled my discussion a few months back with Dr. Hal Thorsrud of Anges Scott College in Georgia. Professor Thorsrud teaches a novel philosophy course at this Liberal Arts institution on the “Introduction to Artificial Intelligence.” Similar to Horowitz, Thorsrud envisions an automated world whereby his graduates would be critical in thinking through the ethical applications of robots and AI. “Ethics has to become an engineering problem which is fascinating because, I mean, the idea is that we need to figure out how we can actually encode our ethical values into these systems. So they will abide by our values in order to pursue what we deemed to be good,” remarked Thorsrud.

According to Thorsrud’s syllabus, the class begins: “with a brief survey of positions in the philosophy of mind in order to better understand the concept of intelligence and to formulate the default position of most AI research, namely Computationalism. We then examine questions such as ‘What is a computer?’, ‘What makes a function or number computable?’, ‘What are algorithms and how do they differ from heuristics?’ We will consider fundamental issues in AI such as what makes a system intelligent, and whether computers can have minds. Finally, we will explore some of the ethical challenges that face AI such as whether intelligent artificial systems should be relied upon to make important decisions that affect our lives, and whether we should create such systems in the first place.”

In explaining the origins of his course, Dr. Thorsrud recalled, “A lot of my students are already interested in philosophy. They just don’t know it. And so, in fact, just recently my department has joined forces with neuroscience. We’re no longer a Philosophy Department. We’re now the Philosophy of Neuroscience, and now the Department of Law, Neuroscience, and Philosophy. Because these students are interested in mind, they are interested in intelligence, but they don’t realize that philosophy has been dealing with an attempt to understand the nature of mind from the very beginning, and the nature of intelligence from the very beginning. So we have a lot to offer these students this question of how to reach them. So that’s what kind of started me off on this different path, and, in the meantime think the same is true of artificial intelligence.”

Thorsrud elaborated that this introductory course is only the first step in a wider AI curriculum at Anges Scott as the confluence between endeavors like Viam and ChatGPT collide in the coming years to move the automation industry at hyperspeed. Already, the AI Philosopher sees how GPT is challenging humans to stand out, “The massive growth in the training data and the parameters, the weights that were that the machine learning was operating on really paid off.” He continued to illustrate how dystopian fears are unfounded, “I mean, we have a tendency to anthropomorphize things like ChatGPT and it’s understandable. But as far as I can tell, it’s, it’s a long way from the intelligence of my dog, a long, long way.” He is realistic about the speed of adoption, “Well, as a philosopher, they’re never going to be able to get to the point where they can give me a credible adjudication, because human judgment you know it is. And, this is another example of the ever-present receding horizon problem. First, you know that computers will never be able to beat a human at chess. Okay, fine computers will never be able to beat a human at GO. Fine computers will never be able to write. And so we keep setting these limits down, and then surpassing them.”

At Automate, I had the chance to catch up with ff Venture Capital portfolio company, PlusOne Robotics, and its amazing founder, Erik Nieves. While the talk in the theater was about the future, Nieves illustrated on the floor what is happening today. Impressively the startup is close to one million picks of depalletizing packages and sorting goods for the likes of FedEx and other leading providers of shipping & logistics. PlusOne’s proprietary computer vision co-bot platform is not waiting for the next generation of developers to join the ranks, but building its own intelligent protocols to increase efficiencies on the front lines of e-commerce fulfillment.

As Brian Marflak, of FedEx, remarked, “The technology in these depalletizing arms helps us move certain shipments that would otherwise take up valuable resources to manually offload. Having these systems installed allows team members to perform more skilled tasks such as loading and unloading airplanes and trucks. This has been a great opportunity for robotics to complement our existing team members and help them complete tasks more efficiently.”

Markflak’s sentiment was shared by the 25,000+ attendees of Automate that filled the entire Detroit Convention Center. A big backdrop of the show was how macro labor trends and shortages are exasperating the push towards automation (and thus moving the horizon even further). According to the most recent reports, close to 20% of all US retail sales are driven online, with over 20 billion packages being shipped every year growing at an annual rate of 25%. This means even if the e-commerce industry is able to hire a million more workers, there are not enough (organic) hands to keep up. As Nieves puts it, “The growth of e-commerce has placed tremendous pressure on shipping responsiveness and scalability that has significantly exacerbated labor and capacity issues. Automation is key, but keeping a human-in-the-loop is essential to running a business 24/7 with greater speed and fewer errors. With the ongoing labor shortages, I believe we’ll see an increase in the adoption of Robots-as-a-Service (RaaS) to lower capital expenditures and deploy automation on a subscription basis.” Get ready for Automate 2024, as the convention moves for the first time to an annual gathering!

View Details

During the 2020 pandemic, members of the reddit & discord r/robotics community rallied to organize an online showcase for members of our community. What was originally envisioned as a small, intimate afternoon video call turned out to be a two day event of participants from across the world. The 2021 and 2022 events showcased a multitude of fantastic projects from the r/Robotics Reddit community, as well as academia and industry.

This year’s event features many wonderful robots including…

• Acrobot, the acrobatic robot!
• A large pneumatic spider covered in flowers, The Flower Spider!
• One of the best DIY arms you’ve ever seen, CM6
• An autonomous Sailboat from the aptly named Oshen!

ProgramAll times are recorded in Eastern Daylight Time (EDT), UTC-4. Check out the full program in our website for more details.

Saturday, 10th of June
Session 1: Robot Arms
10:00 – 11:00 KUKA Research and Development
11:00 – 11:30 Harrison Low – Juggling Robot
11:30 – 11:45 Jan Veverak Koniarik – Open Source Servo Firmware
11:45 – 12:00 Rafael Diaz – Soft Robot Tentacle
12:00 – 12:30 Petar Crnjak – DIY 6-Axis Robot Arm
Lunch Break

Session 2: Social, Domestic, and Hobbyist Robots
14:00 – 15:00 Eliot Horowitz (CEO of VIAM) – The Era of Robotics Unicorns
15:00 – 15:30 Niranj S – Mini Humanoid Robot
15:30 – 15:45 Tommy Hedlund – Interactive Robot with ChatFPT
15:45 – 16:00 Emilie Kroeger – ChatGPT Integration for the Pepper Robot
16:00 – 16:15 Matt Vella – Retrofitting an Omnibot 2000 with a Raspberry Pi
16:15 – 16:30 Keegan Neave – NE-Five Mk3
16:30 – 17:00 Dan Nicholson – Open Source Companion Robot

Sunday, 11th of June
Session 1: Autonomous Mobile Robots
10:00 – 11:00 Keynote TBD
11:00 – 11:30 Ciaran Dowdson – “Sailing into the Future: Oshen’s Mini, Autonomous Robo-Vessels for Enhanced Ocean Exploration”
11:30 – 12:00 James Clayton – Giant, Walking Spider Suit with Real Flowers
12:00 – 12:15 Jacob David Cunningham – SLAM by Blob Tracking and Inertial Tracking
12:15 – 12:30 Carl Draper – Mobile UGV Platform Based on ROS2
12:30 – 12:45 Daniel Strabley – Nightcrawler Tactical Robot
12:45 – 13:15 Saksham Sharma – Multi-Robot Path Planning Using Priority Based Algorithm
Lunch Break

Session 2: Startup & Solutions
14:00 – 15:00 Carter Schultz (AMP Robotics) – The Reality of Robotic Systems
15:00 – 15:15 Jakub Matyszczak – MAB Robotics
15:15 – 15:45 Daniel Simu – Acrobot, the Acrobatic Robot
15:45 – 16:00 Luis Guzman – Zeus2Q, the Humanoid Robotic Platform
16:00 – 16:30 Kshitij Tiwari – The State of Robotic Touch Sensing
16:30 – 16:45 Sayak Nandi – ROS Robots as a Web Application
16:45 – 17:00 Ishant Pundir – Asper and Osmos: A Personal Robot and AI-Based OS

Links* YouTube livestream. * Reddit Robotics subchannel. * Reddit Robotics blog.

View Details

Earlier this spring, the largest robotics event in Europe – The European Robotics Forum 2023 (ERF23) – was held in Odense, Denmark. As one of the most influential gatherings of the robotics community in Europe, the event brought together researchers, engineers, managers, entrepreneurs, businesspeople, and public funding officers to explore the latest trends and themes in the field of robotics. With more than 1100 registered participants and 65 sponsors and exhibitors, this was ‘the largest ERF in recorded history – on all parameters’, say the organizers.

During the four-day forum, RI4EU robotics DIHs network, together with agROBOfood and Rima Network, hosted a booth at the event, where they showcased a range of robotics initiatives. These also included TRINITY Robotics DIHs, agROBOfood, DIH-HERO, and DIH² robotics networks.

One of the highlights of the conference for RI4EU was their workshop – “Supporting SMEs in Bringing Robotics Solutions to Market“. The aim was to have an interactive discussion on how robotics Digital Innovation Hubs (DIHs) networks can create a greater impact for SMEs and facilitate a broad uptake and integration of robotics technologies in the industry. The question comes in the context where a group of 5 EU-funded robotics projects – robotics DIHs networks (agROBOfood, Rima Network, TRINITY, DIH-HERO, and DIH²) – under the umbrella of RI4EU, have provided financial support of 40M EUR and additional services to more than 180 European robotics SMEs, to help them bring their solutions to the market. So now, when these projects are ending after a period of 4 years, it is only natural to discuss the impact they have created, challenges and lessons learned.

The session featured five expert speakers: Minna Lanz, Coordinator of TRINITY Robotics DIHs, Christophe Leroux, Coordinator of Rima Network, Françoise Siepel, Coordinator of DIH-HERO, Ali Muhammad, Coordinator of DIH², and Tsampikos Kounalakis, Robotics Researcher-agROBOfood. Maurits Butter, RI4EU Gateway to EU Robotics Initiatives, was a moderator, ensuring a productive and engaging conversation.

Building trusting relationships is keyThere is massive potential for robotic applications in the industry, for example, to increase productivity, improve safety, etc. but the total market size of robotics is still negligible in relation to the overall market size. The feedback received during the workshop indicated that one of the major issues robot SMEs face is the effort needed to develop Proof of concepts and run tests. Along the same line of reasoning, companies agreed that the 4 most important services they need from DIHs are:

• Technological support, to provide technological infrastructure and expertise to develop the innovation.
• Ecosystem services, aiming at the provision of support to create a dynamic ecosystem.
• Business support, to provide more single customer support on developing an innovation-based business
• Skills and educational support, to enhance the expertise, skills and human resources with the network partners.

However, there was a common key element that companies recognized to have great impact on their activity: the trusted business connections that were facilitated by the DIHs. Building a community of interconnected experts in the many robotics fields requires strategic planning, resources, and determination. So in this sense, DIHs networks can provide opportunities that cannot be found elsewhere. Moreover, scouting for new regional network connections, seeking to connect the value chain, etc. can be a daunting and time-consuming task but then the DIHs networks really bring added value through their connections with other projects or professionals within niche markets, which provide the robotics ecosystem with an entire list of contacts to seeks guidance from or do business with.

Long-term growthBut now, the 5 robotics projects, agROBOfood, Rima Network, TRINITY, DIH-HERO, and DIH², are coming to an end, as they have received funding from the European Union’s Horizon 2020 so far. So what will be the future of these DIHs networks? Will they stop their activity once the EU funding will have ended or will they continue to provide their services in the future? When asked these questions during the workshop, all networks made it clear that they ‘will continue to live’, as one of the speakers said. They are well-established networks in the field of robotics and although their new business models are not officialized yet, the DIHs networks are planning to grow in the future.
So far, the Rima Network has made its announcement: it is now transforming into the RIMA Alliance, to keep providing services and seize all good practices in just one-stop-shop, to keep working towards the uptake of robotics in inspection and maintenance. As for the other 4 networks, follow their pages and RI4EU network to hear fresh news and subscribe to the newsletter on their website.

Through participation in the ERF23, the RI4EU team was able to learn from experts in the field, make new connections, and promote their work to a wider audience. By partnering with their innovation actions/DIHs networks, they were able to showcase their initiatives and inspire others to join them in their mission to accelerate innovation in robotics.

View Details

Researchers created “FluidLab,” a simulation environment with a diverse set of manipulation tasks involving complex fluid dynamics. Image: Alex Shipps/MIT CSAIL via Midjourney

Imagine you’re enjoying a picnic by a riverbank on a windy day. A gust of wind accidentally catches your paper napkin and lands on the water’s surface, quickly drifting away from you. You grab a nearby stick and carefully agitate the water to retrieve it, creating a series of small waves. These waves eventually push the napkin back toward the shore, so you grab it. In this scenario, the water acts as a medium for transmitting forces, enabling you to manipulate the position of the napkin without direct contact.

Humans regularly engage with various types of fluids in their daily lives, but doing so has been a formidable and elusive goal for current robotic systems. Hand you a latte? A robot can do that. Make it? That’s going to require a bit more nuance.

FluidLab, a new simulation tool from researchers at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), enhances robot learning for complex fluid manipulation tasks like making latte art, ice cream, and even manipulating air. The virtual environment offers a versatile collection of intricate fluid handling challenges, involving both solids and liquids, and multiple fluids simultaneously. FluidLab supports modeling solid, liquid, and gas, including elastic, plastic, rigid objects, Newtonian and non-Newtonian liquids, and smoke and air.

At the heart of FluidLab lies FluidEngine, an easy-to-use physics simulator capable of seamlessly calculating and simulating various materials and their interactions, all while harnessing the power of graphics processing units (GPUs) for faster processing. The engine is “differential,” meaning the simulator can incorporate physics knowledge for a more realistic physical world model, leading to more efficient learning and planning for robotic tasks. In contrast, most existing reinforcement learning methods lack that world model that just depends on trial and error. This enhanced capability, say the researchers, lets users experiment with robot learning algorithms and toy with the boundaries of current robotic manipulation abilities.

To set the stage, the researchers tested said robot learning algorithms using FluidLab, discovering and overcoming unique challenges in fluid systems. By developing clever optimization methods, they’ve been able to transfer these learnings from simulations to real-world scenarios effectively.

“Imagine a future where a household robot effortlessly assists you with daily tasks, like making coffee, preparing breakfast, or cooking dinner. These tasks involve numerous fluid manipulation challenges. Our benchmark is a first step towards enabling robots to master these skills, benefiting households and workplaces alike,” says visiting researcher at MIT CSAIL and research scientist at the MIT-IBM Watson AI Lab Chuang Gan, the senior author on a new paper about the research. “For instance, these robots could reduce wait times and enhance customer experiences in busy coffee shops. FluidEngine is, to our knowledge, the first-of-its-kind physics engine that supports a wide range of materials and couplings while being fully differentiable. With our standardized fluid manipulation tasks, researchers can evaluate robot learning algorithms and push the boundaries of today’s robotic manipulation capabilities.”

Fluid fantasiaOver the past few decades, scientists in the robotic manipulation domain have mainly focused on manipulating rigid objects, or on very simplistic fluid manipulation tasks like pouring water. Studying these manipulation tasks involving fluids in the real world can also be an unsafe and costly endeavor.

With fluid manipulation, it’s not always just about fluids, though. In many tasks, such as creating the perfect ice cream swirl, mixing solids into liquids, or paddling through the water to move objects, it’s a dance of interactions between fluids and various other materials. Simulation environments must support “coupling,” or how two different material properties interact. Fluid manipulation tasks usually require pretty fine-grained precision, with delicate interactions and handling of materials, setting them apart from straightforward tasks like pushing a block or opening a bottle.

FluidLab’s simulator can quickly calculate how different materials interact with each other.

Helping out the GPUs is “Taichi,” a domain-specific language embedded in Python. The system can compute gradients (rates of change in environment configurations with respect to the robot’s actions) for different material types and their interactions (couplings) with one another. This precise information can be used to fine-tune the robot’s movements for better performance. As a result, the simulator allows for faster and more efficient solutions, setting it apart from its counterparts.

The 10 tasks the team put forth fell into two categories: using fluids to manipulate hard-to-reach objects, and directly manipulating fluids for specific goals. Examples included separating liquids, guiding floating objects, transporting items with water jets, mixing liquids, creating latte art, shaping ice cream, and controlling air circulation.

“The simulator works similarly to how humans use their mental models to predict the consequences of their actions and make informed decisions when manipulating fluids. This is a significant advantage of our simulator compared to others,” says Carnegie Mellon University PhD student Zhou Xian, another author on the paper. “While other simulators primarily support reinforcement learning, ours supports reinforcement learning and allows for more efficient optimization techniques. Utilizing the gradients provided by the simulator supports highly efficient policy search, making it a more versatile and effective tool.”

Next stepsFluidLab’s future looks bright. The current work attempted to transfer trajectories optimized in simulation to real-world tasks directly in an open-loop manner. For next steps, the team is working to develop a closed-loop policy in simulation that takes as input the state or the visual observations of the environments and performs fluid manipulation tasks in real time, and then transfers the learned policies in real-world scenes.

The platform is publicly publicly available, and researchers hope it will benefit future studies in developing better methods for solving complex fluid manipulation tasks.

“Humans interact with fluids in everyday tasks, including pouring and mixing liquids (coffee, yogurts, soups, batter), washing and cleaning with water, and more,” says University of Maryland computer science professor Ming Lin, who was not involved in the work. “For robots to assist humans and serve in similar capacities for day-to-day tasks, novel techniques for interacting and handling various liquids of different properties (e.g. viscosity and density of materials) would be needed and remains a major computational challenge for real-time autonomous systems. This work introduces the first comprehensive physics engine, FluidLab, to enable modeling of diverse, complex fluids and their coupling with other objects and dynamical systems in the environment. The mathematical formulation of ‘differentiable fluids’ as presented in the paper makes it possible for integrating versatile fluid simulation as a network layer in learning-based algorithms and neural network architectures for intelligent systems to operate in real-world applications.”

Gan and Xian wrote the paper alongside Hsiao-Yu Tung a postdoc in the MIT Department of Brain and Cognitive Sciences; Antonio Torralba, an MIT professor of electrical engineering and computer science and CSAIL principal investigator; Dartmouth College Assistant Professor Bo Zhu, Columbia University PhD student Zhenjia Xu, and CMU Assistant Professor Katerina Fragkiadaki. The team’s research is supported by the MIT-IBM Watson AI Lab, Sony AI, a DARPA Young Investigator Award, an NSF CAREER award, an AFOSR Young Investigator Award, DARPA Machine Common Sense, and the National Science Foundation.

The research was presented at the International Conference on Learning Representations earlier this month.

• PAPER – FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation. Zhou Xian, Bo Zhu, Zhenjia Xu, Hsiao-Yu Tung, Antonio Torralba, Katerina Fragkiadaki, and Chuang Gan. arXiv preprint arXiv:2303.02346 (2023).

View Details

Claire chatted to Elena De Momi from the the Polytechnic University of Milan all about surgical robotics, artificial intelligence, and the upcoming ICRA robotics conference in London.

Elena De Momi received her MSc in Biomedical Engineering in 2002, PhD in Bioengineering in 2006, and she is currently Associate Professor in the Electronic Information and Bioengineering Department (DEIB) of Politecnico di Milano. She is co-founder of the Neuroengineering and Medical Robotics Laboratory, in 2008, being responsible of the Medical Robotics section. Her academic interests include computer vision and image-processing, artificial intelligence, augmented reality and simulators, teleoperation, haptics, medical robotics, human robot interaction.

View Details

There was a shortage of entries in the tablebot competition shortly before the registration window closed for RoboGames 2023. To make sure the contest would be held, I entered a robot. Then I had to build one.

What’s a tablebot?A tablebot lives on the table. There are three “phases” to the competition:

• Phase I: Build a robot that goes from one end of a table to the other and back.
• Phase II: Have the robot push a block off the ledge of the table.
• Phase III: Have the robot push the block into a shoebox mounted at the end of the table.

There is also an unofficial Phase IV – which is to fall off the table and survive. I did not attempt this phase.

The majority of tablebots are quite simple – a couple of sonar or IR sensors and they kind of wander around the tabletop in hopes of completing the different phases. My tablebot is decidedly different – and it paid off as the robot won the gold medal at RoboGames 2023.

Robot buildThe entire robot is built of 3D printed parts and random things I had on hand.

I’ve had one of those $99 LD-06 lidars sitting around for a while, and decided this was a great project to use it on. I used a Dynamixel AX-12 servo to tilt the laser so I can find the table, the cube, or the goal.

All of the code runs on an STM32, on my custom Etherbotix board which was designed for my Maxwell robot a number of years ago. The robot uses differential drive with some 30:1 12V gear motors, which were purchased from Lynxmotion in 2008 and used in various fire fighting robots over the years.

A set of small digital Sharp IR sensors are used as cliff sensors. These can be moved up or down to calibrate for different table surfaces using a pair of adjustment screws. While the sensors are very accurate and stop the robot, they don’t see far enough ahead when going at full speed, and so I also use the laser to detect when the table edge is approaching.

Phase 1 SoftwarePhase 1 is pretty straight forward – and mostly based on dead reckoning odometry:

• The laser is angled downwards looking for the table. This is done by projecting to the scan to 3D points, and filtering out anything not in front of the robot at roughly table height. When the table disappears (number of points drops too low), we reduce our maximum speed to something that is safe for the cliff sensors to detect.
• While the laser sensors look for the end of the table, the robot drives forward, and a simple feedback loop keeps the robot centered on the table using odometry.
• When the cliff sensors eventually trigger, the robot stops, backs up 15 centimeters, and then turns 180 degrees – all using dead reckoning odometry.
• The maximum speed is then reset and we take off to the other end of the table with the same behavior.

Phase 2 SoftwareThe movements of Phase 2 are basically the same as Phase 1 – we drive forward, staying centered with odometry. The speed is a bit lower than Phase 1 because the laser is also looking for the block:

• The laser scan is projected to 3D, and we filter out any points that are part of the table based on height. These remaining points are then clustered and the clusters are analyzed for size.
• If a cluster is a good candidate for the block, the robot turn towards the block (using, you guessed it, dead reckoning from odometry).
• The robot then drives towards the block using a simple control loop to keep the heading.
• Once the block is arrived at, the robot drives straight until a cliff sensor trips.
• At that point, the robot stops the wheel on the side of the tripped cliff sensor and drives the other wheel very slowly forward so that we align the front of the robot with the edge of the table – ensuring the block has been pushed off the table.

Phase 3 SoftwareThe final phase is the most complex, but not by much. As with the earlier phases, the robot moves down the table finding the block:

• Unlike in Phase 2, the robot actually approaches a pose just behind the block.
• Once that pose has been reached, the robot tilts the laser back to level and finds the goal.
• The robot then turns towards the goal in the same way it first turned towards the block.
• The robot then approaches the goal using the same simple control loop, and in the process ends up pushing the block to the goal.

All of the software for my Tablebot is availble on GitHub.

Robogames videoJim Dinunzio, a member of the Homebrew Robotics Club, took a video during the actual competition at Robogames so you can actually see the winning set of runs:

VisualizationTo make development easier, I also wrote a Python GUI that renders the table, the robot odometry trail, the laser data, and detected goals and cubes.

Fun with mathAlong the way I actually ran into a bug in the ARM CMSIS DSP library. I used the arm_sin_cos_f32() function to compute my odometry:

arm_sin_cos_f32(system_state.pose_th * 57.2958f, &sin_th, &cos_th); system_state.pose_x += cos_th * d; system_state.pose_y += sin_th * d; system_state.pose_th = angle_wrap(system_state.pose_th + dth);

This function takes the angle (in degrees!) and returns the sine and cosine of the angle using a lookup table and some interesting interpolation. With the visualization of the robot path, I noticed the robot odometry would occasionally jump to the side and backwards – which made no sense.

Further investigation showed that for very small negative angles, arm_sin_cos_f32 returned huge values. I dug deeper into the code and found that there are several different versions out there:

• The version from my older STM32 library, had this particular issue at very small negative numbers. The same bug was still present in the official CMSIS-DSP on the arm account.
• The version in the current STM32 library had a fix for this spot – but that fix then broke the function for an entire quadrant!

The issue turned out to be quite simple:

• The code uses a 512 element lookup table.
• For a given angle, it has to interpolate between the previous and next entry in the table.
• If your angle fell between the 511th entry and the next (which would be the 0th entry due to wrap around) then you used a random value in the next memory slot to interpolate between (and to compute the interpolation). At one point, this resulted in sin(-1/512) returning outrageous values of like 30.

With that bug fixed, odometry worked flawlessly afterwards. As it turned out, I had this same function/bug existing in some brushless motor control code at work.

Robogames wrap upIt is awesome that RoboGames is back! This little robot won’t be making another appearance, but I am starting to work on a RoboMagellan robot for next year.

View Details

With technology for drones far advanced, the next step is to ensure they can fly safely in cities. Image credit: CC0 via Unsplash

The Spanish resort town of Benidorm is known for its sandy beaches with clear waters, a skyline dominated by towering hotels and tourists from northern Europe. But one day in February, it also served as a testing ground for European society’s future with drones.

Since the local economy depends on tourism during the summer, Benidorm is relatively empty in winter – and that’s a plus when it comes to safety while testing unmanned aerial vehicles (UAVs). The tall buildings that dominate the skyline also stand in nicely for those of a big city.

Sun, sea and…satellite signals

In sum, it’s an ideal place to try out new drone technology. And an EU-funded project called DELOREAN has done just that – testing new types of satellite tracking for drones on 9 February.

‘Benidorm’s skyline is quite similar to what you would find in larger cities like, say, New York,’ said Santiago Soley, the project coordinator who is also chief executive officer of Spanish aeronautics-engineering company Pildo Labs. ‘Generally, regulations limit drone flights over dense urban areas. It’s the first time in Europe we did these intense tests in a challenging city environment.’

Drones have been a hyped technology for years, during which the media popularised predictions that such aircraft would soon be used for all kinds of daily services including delivering packages to people’s doorsteps. Yet so far, widespread civilian use has failed to take off.

The bottleneck is safety and the need to demonstrate to city governments that drones can be operated in large numbers in populated areas without being a hazard. If a UAV crashes onto a busy street or into a plane that’s landing or taking off, the result could be severe damage or even deaths.

“Drone technology is getting there.”

– Santiago Soley, DELOREAN

Scientists and companies are now addressing these concerns – and the experiments in Benidorm might hold the key to the future success of drones.

‘Drone technology is getting there – it’s the least of our problems,’ said Soley. ‘What’s more important is to demonstrate how drones would safely be deployed over cities.’

DELOREAN is wrapping up after three years. The main goal was to develop navigation and positioning requirements for urban air services and show how the European Global Navigation Satellite System, or EGNSS, can help.

Non-GPS options

Drones need to know exactly where they are at all times. For that, UAVs currently rely on satellites, mostly the US Global Positioning System, or GPS. Another alternative to GPS is Europe’s Galileo network.

DELOREAN is also testing Galileo’s potential for drones.

While led by Pildo Labs, the project has featured an international consortium whose members include France-based aircraft manufacturer Airbus, Spanish postal-servicer provider Correos and the European Organisation for the Safety of Air Navigation, or Eurocontrol, in Belgium.

A challenge for satellite tracking in urban areas is that signals might be deflected or otherwise hindered by buildings. Galileo will help avoid such disruptions because of the waveform and structure of its signals, according to Soley.

In addition, Galileo is pioneering new services that could pinpoint drones’ locations with higher accuracy – something DELOREAN tested in Benidorm.

Furthermore, Galileo adds a layer of security. An authentication service that allows the drone to verify whether the satellite signal is real would counter any future efforts by criminal groups to misdirect UAVs and steal their contents through fake signals, according to Soley.

Airborne parcel deliveries

If experiments of the kinds conducted by DELOREAN prove successful, many applications could open up.

“Before businesses like urban air delivery can develop, we first need safety.”

– Professor Luis Moreno Lorente, LABYRINTH

While drones are already in use over cities, it is often in small-scale operations by local authorities. Police departments, for one, use them to monitor crowds or track speeding cars.

‘There are limitations on drone flights and you need to close the area,’ said Soley. ‘At the technical level, however, the flights are quite easy to handle.’

The next step could be mass urban air delivery. No more vans zigzagging through city streets with all the congestion and pollution.

Instead, fleets of drones would drop off packages across town. Companies like Amazon are already rolling out these services in limited areas.

‘Logistics will, I think, be one of the most promising uses of drones,’ said Soley.

Self-flying craft

An EU-funded project called LABYRINTH is tackling the challenge of ensuring that autonomous drones keep track of each other.

An ARQUIMEA drone being tested in Marugán, Segovia, Spain. © Labyrinth, 2023

Autonomous drones require no ground-based human pilots, who are generally needed for the current generation of UAVs.

‘In the future, those drones will be operated autonomously – they will fly themselves,’ said Luis Moreno Lorente, the project coordinator and a professor of systems engineering and automation at the University Carlos III of Madrid in Spain. ‘But if you want to do that safely, you need to know exactly where each one of them is located.’

LABYRINTH, which is due to end in May after three years, is developing software that acts as an air traffic control system for drones. The 3D position of each is tracked and the aircraft then relays this information to other drones in the vicinity so they don’t crash into each other.

Similarly, if a drone faces technical troubles – say one of its motors fails – it needs to be able to direct other UAVs away from it.

‘Before businesses like urban air delivery can develop, we first need safety,’ said Moreno Lorente. ‘That’s what we’re building now.’

Together, LABYRINTH and DELOREAN are helping to clear the way for a future in which large numbers of drones fly over cities.

‘It’s just a matter of time before they do,’ said Moreno Lorente.

Watch the video

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

Credit: Thomas Hartung, Johns Hopkins University

By Liad Hollender, Frontiers science writer

Despite AI’s impressive track record, its computational power pales in comparison with that of the human brain. Scientists unveil a revolutionary path to drive computing forward: organoid intelligence (OI), where lab-grown brain organoids serve as biological hardware. “This new field of biocomputing promises unprecedented advances in computing speed, processing power, data efficiency, and storage capabilities – all with lower energy needs,” say the authors in an article published in Frontiers in Science.

Artificial intelligence (AI) has long been inspired by the human brain. This approach proved highly successful: AI boasts impressive achievements – from diagnosing medical conditions to composing poetry. Still, the original model continues to outperform machines in many ways. This is why, for example, we can ‘prove our humanity’ with trivial image tests online. What if instead of trying to make AI more brain-like, we went straight to the source?

Scientists across multiple disciplines are working to create revolutionary biocomputers where three-dimensional cultures of brain cells, called brain organoids, serve as biological hardware. They describe their roadmap for realizing this vision in the journal Frontiers in Science.

“We call this new interdisciplinary field ‘organoid intelligence’ (OI),” said Prof Thomas Hartung of Johns Hopkins University. “A community of top scientists has gathered to develop this technology, which we believe will launch a new era of fast, powerful, and efficient biocomputing.”

What are brain organoids, and why would they make good computers?Brain organoids are a type of lab-grown cell-culture. Even though brain organoids aren’t ‘mini brains’, they share key aspects of brain function and structure such as neurons and other brain cells that are essential for cognitive functions like learning and memory. Also, whereas most cell cultures are flat, organoids have a three-dimensional structure. This increases the culture’s cell density 1,000-fold, meaning that neurons can form many more connections.

But even if brain organoids are a good imitation of brains, why would they make good computers? After all, aren’t computers smarter and faster than brains?

“While silicon-based computers are certainly better with numbers, brains are better at learning,” Hartung explained. “For example, AlphaGo [the AI that beat the world’s number one Go player in 2017] was trained on data from 160,000 games. A person would have to play five hours a day for more than 175 years to experience these many games.” 

Brains are not only superior learners, they are also more energy efficient. For instance, the amount of energy spent training AlphaGo is more than is needed to sustain an active adult for a decade.

“Brains also have an amazing capacity to store information, estimated at 2,500TB,” Hartung added. “We’re reaching the physical limits of silicon computers because we cannot pack more transistors into a tiny chip. But the brain is wired completely differently. It has about 100bn neurons linked through over connection points. It’s an enormous power difference compared to our current technology.”

What would organoid intelligence bio computers look like?According to Hartung, current brain organoids need to be scaled-up for OI. “They are too small, each containing about 50,000 cells. For OI, we would need to increase this number to 10 million,” he explained.

In parallel, the authors are also developing technologies to communicate with the organoids: in other words, to send them information and read out what they’re ‘thinking’. The authors plan to adapt tools from various scientific disciplines, such as bioengineering and machine learning, as well as engineer new stimulation and recording devices.

“We developed a brain-computer interface device that is a kind of an EEG cap for organoids, which we presented in an article published last August. It is a flexible shell that is densely covered with tiny electrodes that can both pick up signals from the organoid, and transmit signals to it,” said Hartung.

The authors envision that eventually OI would integrate a wide range of stimulation and recording tools. These will orchestrate interactions across networks of interconnected organoids that implement more complex computations.

Organoid intelligence could help prevent and treat neurological conditionsOI’s promise goes beyond computing and into medicine. Thanks to a groundbreaking technique developed by Noble Laureates John Gurdon and Shinya Yamanaka, brain organoids can be produced from adult tissues. This means that scientists can develop personalized brain organoids from skin samples of patients suffering from neural disorders, such as Alzheimer’s disease. They can then run multiple tests to investigate how genetic factors, medicines, and toxins influence these conditions.

“With OI, we could study the cognitive aspects of neurological conditions as well,” Hartung said. “For example, we could compare memory formation in organoids derived from healthy people and from Alzheimer’s patients, and try to repair relative deficits. We could also use OI to test whether certain substances, such as pesticides, cause memory or learning problems.”

Taking ethical considerations into accountCreating human brain organoids that can learn, remember, and interact with their environment raises complex ethical questions. For example, could they develop consciousness, even in a rudimentary form? Could they experience pain or suffering? And what rights would people have concerning brain organoids made from their cells?

The authors are acutely aware of these issues. “A key part of our vision is to develop OI in an ethical and socially responsible manner,” Hartung said. “For this reason, we have partnered with ethicists from the very beginning to establish an ‘embedded ethics’ approach. All ethical issues will be continuously assessed by teams made up of scientists, ethicists, and the public, as the research evolves.”

How far are we from the first organoid intelligence?Even though OI is still in its infancy, a recently-published study by one of the article’s co-authors – Dr Brett Kagan, Chief Scientific Officer at Cortical Labs – provides proof of concept. His team showed that a normal, flat brain cell culture can learn to play the video game Pong.

“Their team is already testing this with brain organoids,” Hartung added. “And I would say that replicating this experiment with organoids already fulfills the basic definition of OI. From here on, it’s just a matter of building the community, the tools, and the technologies to realize OI’s full potential,” he concluded.

Interview with Prof Thomas HartungImage: Prof Thomas Hartung

To learn more about this exciting new field, we interviewed the senior author of the article, Prof Thomas Hartung. He is the director of the Center for Alternatives to Animal Testing in Europe (CAAT-Europe), and a professor at Johns Hopkins University’s Bloomberg School of Public Health.

How do you define organoid intelligence?

Reproducing cognitive functions – such as learning and sensory processing – in a lab-grown human-brain model.

How did this idea emerge?

I’m a pharmacologist and toxicologist, so I’m interested in developing medicines and identifying substances that are dangerous to our health, specifically those that affect brain development and function. This requires testing – ideally in conditions that mimic a living brain. For that reason, producing cultures of human brain cells has been a longstanding aim in the field.

This goal was finally realized in 2006 thanks to a groundbreaking technique developed by John B. Gurdon and Shinya Yamanaka, who received a Nobel prize for this achievement in 2012. This method allowed us to generate brain cells from fully developed tissues, such as the skin. Soon after, we began mass producing three-dimensional cultures of brain cells called brain organoids.

People asked if the organoids were thinking, if they were conscious even. I said: “no, they are too tiny. And more importantly, they don’t have any input nor output, so what would they be thinking about?” But later I began wondering: what if we changed this? What if we gave the organoids information about their environment and the means to interact with it? That was the birth of organoid intelligence.

How would you know what an organoid is ‘thinking’ about?

We’re building tools that will enable us to communicate with the organoids – send input and receive output. For example, we developed a recording/stimulation device that looks like a mini EEG-cap that surrounds the organoid. We’ve also been working on feeding biological inputs to brain organoids, for instance, by connecting them to retinal organoids, which respond to light. Our partner and co-author Alysson Muotri at the University of San Diego is already testing this approach by producing systems that combine several organoids.

My dream is to form a channel of communication between an artificial intelligence program and an OI system that would allow the two to explore each other’s capabilities. I imagine that form will follow function – that the organoid will change and develop towards creating meaningful inputs. This is a bit of philosophy, but my expectation is that we’ll see a lot of surprises.
What uses do you envision for organoid intelligence?

In my opinion, there are three main areas. The first is fundamental neuroscience – to understand how the brain generates cognitive functions, such as learning and memory. Even though current brain organoids are still far from being what one might call intelligent, they could still have the machinery to support basic cognitive operations.

The second area is toxicology and pharmacology. Since we can now produce brain organoids from skin samples, we can study individual disease characteristics of patients. We already have brain-organoid lines from Alzheimer’s patients, for example. And even though these organoids were made from skin cells, we still see hallmarks of the disease in them.

Next, we would like to test if there are also differences in their memory function, and if so, if we could repair it. We can also test whether substances, such as pesticides, worsen cognitive deficits, or cause them in brain organoids produced from healthy subjects. This is a very exciting line of research, which I believe is nearly within reach.

The third area is computing. As we laid out in our article, considering the brain’s size, its computational power is simply unmatched. Just for comparison, a supercomputer finally surpassed the computational power of a single human brain in 2022. But it cost $600m and occupies 680 square meters [about twice the area of a tennis court].

We’re also reaching the limits of computing. Moore’s Law, which states that the number of transistors in a microchip doubles every two years, has held for 60 years. But soon we won’t be able to physically fit more transistors into a chip. A single neuron, on the other hand, can connect to up to 10,000 other neurons – this is a very different way of processing and storing information. Through OI, we hope that we’ll be able to leverage the brain’s computational principles to build computers differently.

How do you intend to tackle ethical issues that might arise from organoid intelligence?

There are many questions that we face now, ranging from the rights of people over organoids developed from their cells, to understanding whether OI is conscious. I find this aspect of the work fascinating, and I believe it’s a fantastic opportunity to investigate the physical manifestation of concepts like sentience and consciousness.

We teamed up with Jeffrey Kahn of the Bloomberg School of Public Health at Johns Hopkins University at the very beginning, asking him to lead the discussion around the ethics of neural systems. We have come up with two main strategies. One is called embedded ethics: we want ethicists to closely observe the work, take part in the planning, and raise points early on. The second part focuses on the public – we intend to share our work broadly and clearly as it advances. We want to know how people feel about this technology and define our research plan accordingly.

How far are we from the first organoid intelligence?

Even though OI is still in its infancy, past work shows that it’s possible. A study by one of our partners and co-authors – Brett Kagan of the Cortical Labs – is a recent example. His team showed that a standard brain cell culture can learn to play the video game Pong. They are already experimenting with brain organoids, and I would say that replicating this with organoids already fulfills what we call OI.

Still, we are a long way from realizing OI’s full potential. When it becomes a real tool, it will look very different from these first baby steps we are taking now. The important thing is that it’s a starting point. I see this like sequencing the first genes of the human genome project: the enabling technology is in our hands, and we’re bound to learn a lot on the way.

This post is a combination of the original articles published on the Frontiers in Robotics and AI blog. You can read the originals here and here.

• PAPER – Organoid intelligence (OI): the new frontier in biocomputing and intelligence-in-a-dish. Smirnova, L., Caffo, B.S., Gracias, D.H., Huang, Q., Morales Pantoja, I.E., Tang, B., Zack, D.J., Berlinicke, C.A., Boyd, J.L., Harris, T.D. and Johnson, E.C., Frontiers in Science, 2023.

View Details

Claire chatted to Helmut Hauser from the University of Bristol all about soft robotics, sensing, and smart robot bodies.

Helmut Hauser is an Associate Professor in Robotics at the University of Bristol and the Bristol Robotics Laboratory. He is also the Director of the EPSRC Centre of Doctoral Training for Robotics and Autonomous Systems. Helmut’s research is focused on morphological computation and soft robotics. In particular, he is interested in understanding the underlying principles of how biological systems exploit their complex physical bodies to facilitate sensing, controlling and learning, and how these principles can be employed to design better bodies to build better robots.

View Details

India´s operational stock of industrial robots hit all time high.

Sales of industrial robots in India reached a new record of 4,945 units installed. This is an increase of 54 percent compared to the previous year (2020: 3,215 units). In terms of annual installations, India now ranks in tenth position worldwide. These are findings of the report World Robotics, presented by the International Federation of Robotics (IFR).

“India is one of the world’s fastest-growing industrial economies,” says Marina Bill, President of the International Federation of Robotics. “Within five years, the operational stock of industrial robots has more than doubled, to reach 33,220 units in 2021. This corresponds to an average annual growth rate of 16% since 2016.”

Today, India is the world’s fifth largest economy measured by manufacturing output. According to World Bank data, India´s manufacturing value added in 2021 was USD 443.9 billion, a 21.6% increase from 2020.

The automotive industry remains the largest customer for the robotics industry in India with a share of 31% in 2021. Installations more than doubled to 1,547 units (+108%). The general industry in India is led by the metal industry with 308 units (-9%), the rubber and plastics industry with 246 units (+27%) and the electrical/electronics industry with 215 units (+98%).

Impressive potential for IndiaThe long-term potential of robotics in India becomes clearer when compared to China: India´s robot density in the automotive industry, which is the number of industrial robots per 10,000 employees, reached 148 robots in 2021. China´s robot density hit 131 units in 2010 and skyrocketed to 772 units in 2021.

The Indian government supports growth in the industrial sector as one of the vital figures that affect the Gross Domestic Product (GDP). Today, the country´s GDP of about USD 3 trillion ranks in fifth place, head-to-head with the UK and France – behind Germany, Japan, China and the USA – the International Monetary Fund reports.

Outlook for India“As a result of the recent supply chain disruption, companies are rethinking their nearshoring strategies in Southeast Asia,” says Marina Bill. “India has traditionally been a popular destination for nearshoring in the manufacturing segment. The Indian government wants the country to be considered for new diversification options such as friendshoring, which is partnering with countries that share similar values and interests.”

The manufacturing sector is also expected to benefit from the government’s initiatives to boost its competitiveness and attractiveness for investors. The Production Linked Incentive (PLI) scheme, for example, currently set to run until 2025, subsidizes companies that create production capacity in India in robot customer industries like automotive, metal, pharmaceuticals, and food processing.

Robots help to create new jobsNew manufacturing capacities in India are an important step to provide adequate education and employment opportunities for its people: According to projections of the United Nations, India now has a population of 1,4 billion, surpassing China for the first time. This means that India has a large and young workforce that can drive economic growth and innovation. India is expected to have the largest working-age population in the world by 2027.

View Details

Robot fish. Image credit: Tsam Lung You

The robot fish was fitted with a twisted and coiled polymer (TCP) to drive it forward, a light-weight low cost device that relies on temperature change to generate movement, which also limits its speed.

A TCP works by contracting like muscles when heated, converting the energy into mechanical motion. The TCP used in this work is warmed by Joule heating – the pass of current through an electrical conductor produces thermal energy and heats up the conductor. By minimising the distance between the TCP on one side of the robot fish and the spring on the other, this activates the fin at the rear, enabling the robot fish to reach new speeds. The undulating flapping of its rear fin was measured at a frequency of 2Hz, two waves per second. The frequency of the electric current is the same as the frequency of tail flap.

The findings, published at the 6th IEEE-RAS International Conference on Soft Robotics (RoboSoft 2023), provide a new route to raising the actuation – the action of causing a machine or device to operate – frequency of TCPs through thermomechanical design and shows the possibility of using TCPs at high frequency in aqueous environments.

Lead author Tsam Lung You from Bristol’s Department of Engineering Mathematics said: “Twisted and coiled polymer (TCP) actuator is a promising novel actuator, exhibiting attractive properties of light weight, low-cost high energy density and simple fabrication process.

“They can be made from very easily assessable materials such as a fishing line and they contract and provide linear actuation when heated up. However, because of the time needed for heat dissipation during the relaxation phase, this makes them slow.”

By optimising the structural design of the TCP-spring antagonistic muscle pair and bringing their anchor points closer together, it allowed the posterior fin to swing at a larger angle for the same amount of TCP actuation.

Antagonistic muscles. Image credit: Tsam Lung You

Although this requires greater force, TCP is a strong actuator with high work energy density, and is still able to drive the fin.

Until now, TCPs have been mostly used for applications such as wearable devices and robotic hands. This work opens up more areas of application where TCP can be used, such as marine robots for underwater exploration and monitoring.

Tsam Lung You added: “Our robotic fish swam at the fastest actuation frequency found in a real TCP application and also the highest locomotion speed of a TCP application so far.

“This is really exciting as it opens up more opportunities of TCP application in different areas.”

The team now plan to expand the scale and develop a knifefish-inspired TCP-driven ribbon fin robot that can swim agilely in water.

• PAPER – Robotic Fish driven by Twisted and Coiled Polymer Actuators at High Frequencies.. You, T. L., Rossiter, J. M., and Philamore, H. In 6th IEEE-RAS International Conference on Soft Robotics (RoboSoft).

View Details

This short film documents some of the most innovative projects that emerged from the work of NCCR Robotics, the Swiss-wide consortium coordinated from 2010 to 2022 by EPFL professor Dario Floreano and ETHZ professor Robert Riener, including other major research institutions across Switzerland.

Shot over the course of six months in Lausanne, Geneva, Zurich, Wangen an der Aare, Leysin, Lugano, the documentary is a unique look at the state of the art of medical, educational and rescue robotics, and at the specific contributions that Swiss researchers have given to the field over the last decade. In addition to showing the robots in action, the film features extended interviews with top experts including Stéphanie Lacour, Silvestro Micera, Davide Scaramuzza, Robert Riener, Pierre Dillenbourg, Margarita Chli, Dario Floreano.

Produced by NCCR Robotics and Viven.

View Details

Shutterstock / Frame Stock Footage

By Thusha Rajendran (Professor of Psychology, The National Robotarium, Heriot-Watt University)

The social separation imposed by the pandemic led us to rely on technology to an extent we might never have imagined – from Teams and Zoom to online banking and vaccine status apps.

Now, society faces an increasing number of decisions about our relationship with technology. For example, do we want our workforce needs fulfilled by automation, migrant workers, or an increased birth rate?

In the coming years, we will also need to balance technological innovation with people’s wellbeing – both in terms of the work they do and the social support they receive.

And there is the question of trust. When humans should trust robots, and vice versa, is a question our Trust Node team is researching as part of the UKRI Trustworthy Autonomous Systems hub. We want to better understand human-robot interactions – based on an individual’s propensity to trust others, the type of robot, and the nature of the task. This, and projects like it, could ultimately help inform robot design.

This is an important time to discuss what roles we want robots and AI to take in our collective future – before decisions are taken that may prove hard to reverse. One way to frame this dialogue is to think about the various roles robots can fulfill.

Robots as our servantsThe word “robot” was first used by the Czech writer, Karel Čapek, in his 1920 sci-fi play Rossum’s Universal Robots. It comes from the word “robota”, meaning to do the drudgery or donkey work. This etymology suggests robots exist to do work that humans would rather not. And there should be no obvious controversy, for example, in tasking robots to maintain nuclear power plants or repair offshore wind farms.

The more human a robot looks, the more we trust it. Antonello Marangi/Shutterstock

However, some service tasks assigned to robots are more controversial, because they could be seen as taking jobs from humans.

For example, studies show that people who have lost movement in their upper limbs could benefit from robot-assisted dressing. But this could be seen as automating tasks that nurses currently perform. Equally, it could free up time for nurses and careworkers – currently sectors that are very short-staffed – to focus on other tasks that require more sophisticated human input.

Authority figuresThe dystopian 1987 film Robocop imagined the future of law enforcement as autonomous, privatised, and delegated to cyborgs or robots.

Today, some elements of this vision are not so far away: the San Francisco Police Department has considered deploying robots – albeit under direct human control – to kill dangerous suspects.

This US military robot is fitted with a machine gun to turn it into a remote weapons platform. US Army

But having robots as authority figures needs careful consideration, as research has shown that humans can place excessive trust in them.

In one experiment, a “fire robot” was assigned to evacuate people from a building during a simulated blaze. All 26 participants dutifully followed the robot, even though half had previously seen the robot perform poorly in a navigation task.

Robots as our companionsIt might be difficult to imagine that a human-robot attachment would have the same quality as that between humans or with a pet. However, increasing levels of loneliness in society might mean that for some people, having a non-human companion is better than nothing.

The Paro Robot is one of the most commercially successful companion robots to date – and is designed to look like a baby harp seal. Yet research suggests that the more human a robot looks, the more we trust it.

The Paro companion robot is designed to look like a baby seal. Angela Ostafichuk / Shutterstock

A study has also shown that different areas of the brain are activated when humans interact with either another human or a robot. This suggests our brains may recognise interactions with a robot differently from human ones.

Creating useful robot companions involves a complex interplay of computer science, engineering and psychology. A robot pet might be ideal for someone who is not physically able to take a dog for its exercise. It might also be able to detect falls and remind someone to take their medication.

How we tackle social isolation, however, raises questions for us as a society. Some might regard efforts to “solve” loneliness with technology as the wrong solution for this pervasive problem.

What can robotics and AI teach us?Music is a source of interesting observations about the differences between human and robotic talents. Committing errors in the way humans do all the time, but robots might not, appears to be a vital component of creativity.

A study by Adrian Hazzard and colleagues pitted professional pianists against an autonomous disklavier (an automated piano with keys that move as if played by an invisible pianist). The researchers discovered that, eventually, the pianists made mistakes. But they did so in ways that were interesting to humans listening to the performance.

This concept of “aesthetic failure” can also be applied to how we live our lives. It offers a powerful counter-narrative to the idealistic and perfectionist messages we constantly receive through television and social media – on everything from physical appearance to career and relationships.

As a species, we are approaching many crossroads, including how to respond to climate change, gene editing, and the role of robotics and AI. However, these dilemmas are also opportunities. AI and robotics can mirror our less-appealing characteristics, such as gender and racial biases. But they can also free us from drudgery and highlight unique and appealing qualities, such as our creativity.

We are in the driving seat when it comes to our relationship with robots – nothing is set in stone, yet. But to make educated, informed choices, we need to learn to ask the right questions, starting with: what do we actually want robots to do for us?

Thusha Rajendran receives funding from the UKRI and EU. He would like to acknowledge evolutionary anthropologist Anna Machin’s contribution to this article through her book Why We Love, personal communications and draft review.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

View Details

Claire chatted to Carlotta Berry from the Rose-Hulman Institute of Technology all about robotics education, science outreach, and increasing participation.

Carlotta Berry has a bachelor’s degree in mathematics from Spelman College, bachelor’s degree in electrical engineering from Georgia Institute of Technology, master’s in electrical engineering from Wayne State University, and PhD from Vanderbilt University. Her research interests include robotics education, human-robot interaction, and increasing marginalized and minoritized populations in STEM fields. Berry is a prolific speaker and author of the text, “Mobile Robotics for Multidisciplinary Study” and Black STEM romance novel, “Elevated Inferno: Monet’s Moment”.

View Details

Researchers from MIT and elsewhere have built a wake-up receiver that communicates using terahertz waves, which enabled them to produce a chip more than 10 times smaller than similar devices. Their receiver, which also includes authentication to protect it from a certain type of attack, could help preserve the battery life of tiny sensors or robots. Image: Jose-Luis Olivares/MIT with figure courtesy of the researchers

By Adam Zewe | MIT News Office

Scientists are striving to develop ever-smaller internet-of-things devices, like sensors tinier than a fingertip that could make nearly any object trackable. These diminutive sensors have miniscule batteries which are often nearly impossible to replace, so engineers incorporate wake-up receivers that keep devices in low-power “sleep” mode when not in use, preserving battery life.

Researchers at MIT have developed a new wake-up receiver that is less than one-tenth the size of previous devices and consumes only a few microwatts of power. Their receiver also incorporates a low-power, built-in authentication system, which protects the device from a certain type of attack that could quickly drain its battery.

Many common types of wake-up receivers are built on the centimeter scale since their antennas must be proportional to the size of the radio waves they use to communicate. Instead, the MIT team built a receiver that utilizes terahertz waves, which are about one-tenth the length of radio waves. Their chip is barely more than 1 square millimeter in size.

They used their wake-up receiver to demonstrate effective, wireless communication with a signal source that was several meters away, showcasing a range that would enable their chip to be used in miniaturized sensors.

For instance, the wake-up receiver could be incorporated into microrobots that monitor environmental changes in areas that are either too small or hazardous for other robots to reach. Also, since the device uses terahertz waves, it could be utilized in emerging applications, such as field-deployable radio networks that work as swarms to collect localized data.

“By using terahertz frequencies, we can make an antenna that is only a few hundred micrometers on each side, which is a very small size. This means we can integrate these antennas to the chip, creating a fully integrated solution. Ultimately, this enabled us to build a very small wake-up receiver that could be attached to tiny sensors or radios,” says Eunseok Lee, an electrical engineering and computer science (EECS) graduate student and lead author of a paper on the wake-up receiver.

Lee wrote the paper with his co-advisors and senior authors Anantha Chandrakasan, dean of the MIT School of Engineering and the Vannevar Bush Professor of Electrical Engineering and Computer Science, who leads the Energy-Efficient Circuits and Systems Group, and Ruonan Han, an associate professor in EECS, who leads the Terahertz Integrated Electronics Group in the Research Laboratory of Electronics; as well as others at MIT, the Indian Institute of Science, and Boston University. The research is being presented at the IEEE Custom Integrated Circuits Conference.

Scaling down the receiverTerahertz waves, found on the electromagnetic spectrum between microwaves and infrared light, have very high frequencies and travel much faster than radio waves. Sometimes called “pencil beams,” terahertz waves travel in a more direct path than other signals, which makes them more secure, Lee explains.

However, the waves have such high frequencies that terahertz receivers often multiply the terahertz signal by another signal to alter the frequency, a process known as frequency mixing modulation. Terahertz mixing consumes a great deal of power.

Instead, Lee and his collaborators developed a zero-power-consumption detector that can detect terahertz waves without the need for frequency mixing. The detector uses a pair of tiny transistors as antennas, which consume very little power.

Even with both antennas on the chip, their wake-up receiver was only 1.54 square millimeters in size and consumed less than 3 microwatts of power. This dual-antenna setup maximizes performance and makes it easier to read signals.

Once received, their chip amplifies a terahertz signal and then converts analog data into a digital signal for processing. This digital signal carries a token, which is a string of bits (0s and 1s). If the token corresponds to the wake-up receiver’s token, it will activate the device.

Ramping up securityIn most wake-up receivers, the same token is reused multiple times, so an eavesdropping attacker could figure out what it is. Then the hacker could send a signal that would activate the device over and over again, using what is called a denial-of-sleep attack.

“With a wake-up receiver, the lifetime of a device could be improved from one day to one month, for instance, but an attacker could use a denial-of-sleep attack to drain that entire battery life in even less than a day. That is why we put authentication into our wake-up receiver,” he explains.

They added an authentication block that utilizes an algorithm to randomize the device’s token each time, using a key that is shared with trusted senders. This key acts like a password — if a sender knows the password, they can send a signal with the right token. The researchers do this using a technique known as lightweight cryptography, which ensures the entire authentication process only consumes a few extra nanowatts of power.

They tested their device by sending terahertz signals to the wake-up receiver as they increased the distance between the chip and the terahertz source. In this way, they tested the sensitivity of their receiver — the minimum signal power needed for the device to successfully detect a signal. Signals that travel farther have less power.

“We achieved 5- to 10-meter longer distance demonstrations than others, using a device with a very small size and microwatt level power consumption,” Lee says.

But to be most effective, terahertz waves need to hit the detector dead-on. If the chip is at an angle, some of the signal will be lost. So, the researchers paired their device with a terahertz beam-steerable array, recently developed by the Han group, to precisely direct the terahertz waves. Using this technique, communication could be sent to multiple chips with minimal signal loss.

In the future, Lee and his collaborators want to tackle this problem of signal degradation. If they can find a way to maintain signal strength when receiver chips move or tilt slightly, they could increase the performance of these devices. They also want to demonstrate their wake-up receiver in very small sensors and fine-tune the technology for use in real-world devices.

“We have developed a rich technology portfolio for future millimeter-sized sensing, tagging, and authentication platforms, including terahertz backscattering, energy harvesting, and electrical beam steering and focusing. Now, this portfolio is more complete with Eunseok’s first-ever terahertz wake-up receiver, which is critical to save the extremely limited energy available on those mini platforms,” Han says.

Additional co-authors include Muhammad Ibrahim Wasiq Khan PhD ’22; Xibi Chen, an EECS graduate student; Ustav Banerjee PhD ’21, an assistant professor at the Indian Institute of Science; Nathan Monroe PhD ’22; and Rabia Tugce Yazicigil, an assistant professor of electrical and computer engineering at Boston University.

View Details

Makram Chahine, a PhD student in electrical engineering and computer science and an MIT CSAIL affiliate, leads a drone used to test liquid neural networks. Photo: Mike Grimmett/MIT CSAIL

By Rachel Gordon | MIT CSAIL

In the vast, expansive skies where birds once ruled supreme, a new crop of aviators is taking flight. These pioneers of the air are not living creatures, but rather a product of deliberate innovation: drones. But these aren’t your typical flying bots, humming around like mechanical bees. Rather, they’re avian-inspired marvels that soar through the sky, guided by liquid neural networks to navigate ever-changing and unseen environments with precision and ease.

Inspired by the adaptable nature of organic brains, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have introduced a method for robust flight navigation agents to master vision-based fly-to-target tasks in intricate, unfamiliar environments. The liquid neural networks, which can continuously adapt to new data inputs, showed prowess in making reliable decisions in unknown domains like forests, urban landscapes, and environments with added noise, rotation, and occlusion. These adaptable models, which outperformed many state-of-the-art counterparts in navigation tasks, could enable potential real-world drone applications like search and rescue, delivery, and wildlife monitoring.

The researchers’ recent study, published in Science Robotics, details how this new breed of agents can adapt to significant distribution shifts, a long-standing challenge in the field. The team’s new class of machine-learning algorithms, however, captures the causal structure of tasks from high-dimensional, unstructured data, such as pixel inputs from a drone-mounted camera. These networks can then extract crucial aspects of a task (i.e., understand the task at hand) and ignore irrelevant features, allowing acquired navigation skills to transfer targets seamlessly to new environments.

Drones navigate unseen environments with liquid neural networks.

“We are thrilled by the immense potential of our learning-based control approach for robots, as it lays the groundwork for solving problems that arise when training in one environment and deploying in a completely distinct environment without additional training,” says Daniela Rus, CSAIL director and the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT. “Our experiments demonstrate that we can effectively teach a drone to locate an object in a forest during summer, and then deploy the model in winter, with vastly different surroundings, or even in urban settings, with varied tasks such as seeking and following. This adaptability is made possible by the causal underpinnings of our solutions. These flexible algorithms could one day aid in decision-making based on data streams that change over time, such as medical diagnosis and autonomous driving applications.”

A daunting challenge was at the forefront: Do machine-learning systems understand the task they are given from data when flying drones to an unlabeled object? And, would they be able to transfer their learned skill and task to new environments with drastic changes in scenery, such as flying from a forest to an urban landscape? What’s more, unlike the remarkable abilities of our biological brains, deep learning systems struggle with capturing causality, frequently over-fitting their training data and failing to adapt to new environments or changing conditions. This is especially troubling for resource-limited embedded systems, like aerial drones, that need to traverse varied environments and respond to obstacles instantaneously.

The liquid networks, in contrast, offer promising preliminary indications of their capacity to address this crucial weakness in deep learning systems. The team’s system was first trained on data collected by a human pilot, to see how they transferred learned navigation skills to new environments under drastic changes in scenery and conditions. Unlike traditional neural networks that only learn during the training phase, the liquid neural net’s parameters can change over time, making them not only interpretable, but more resilient to unexpected or noisy data.

In a series of quadrotor closed-loop control experiments, the drones underwent range tests, stress tests, target rotation and occlusion, hiking with adversaries, triangular loops between objects, and dynamic target tracking. They tracked moving targets, and executed multi-step loops between objects in never-before-seen environments, surpassing performance of other cutting-edge counterparts.

The team believes that the ability to learn from limited expert data and understand a given task while generalizing to new environments could make autonomous drone deployment more efficient, cost-effective, and reliable. Liquid neural networks, they noted, could enable autonomous air mobility drones to be used for environmental monitoring, package delivery, autonomous vehicles, and robotic assistants.

“The experimental setup presented in our work tests the reasoning capabilities of various deep learning systems in controlled and straightforward scenarios,” says MIT CSAIL Research Affiliate Ramin Hasani. “There is still so much room left for future research and development on more complex reasoning challenges for AI systems in autonomous navigation applications, which has to be tested before we can safely deploy them in our society.”

“Robust learning and performance in out-of-distribution tasks and scenarios are some of the key problems that machine learning and autonomous robotic systems have to conquer to make further inroads in society-critical applications,” says Alessio Lomuscio, professor of AI safety in the Department of Computing at Imperial College London. “In this context, the performance of liquid neural networks, a novel brain-inspired paradigm developed by the authors at MIT, reported in this study is remarkable. If these results are confirmed in other experiments, the paradigm here developed will contribute to making AI and robotic systems more reliable, robust, and efficient.”

Clearly, the sky is no longer the limit, but rather a vast playground for the boundless possibilities of these airborne marvels.

Hasani and PhD student Makram Chahine; Patrick Kao ’22, MEng ’22; and PhD student Aaron Ray SM ’21 wrote the paper with Ryan Shubert ’20, MEng ’22; MIT postdocs Mathias Lechner and Alexander Amini; and Daniela Rus.

This research was supported, in part, by Schmidt Futures, the U.S. Air Force Research Laboratory, the U.S. Air Force Artificial Intelligence Accelerator, and the Boeing Co.

• PAPER – Robust flight navigation out of distribution with liquid neural networks. Makram Chahine, Ramin Hasani, Patrick Kao, Aaron Ray, Ryan Shubert, Mathias Lechner, Alexander Amini, and Daniela Rus. Science Robotics, 8(77), eadc8892.

View Details

Claire chatted to Francesco Giorgio-Serchi from the University of Edinburgh all about underwater robots, weather-proofing, and soft robotics.

Francesco Giorgio-Serchi is a Lecturer and Chancellor’s Fellow in Robotics and Autonomous Systems at the University of Edinburgh. His work encompasses the design and control of underwater vehicles for operation in extreme weather conditions. Previously he was a Research Fellow at the University of Southampton, within the Fluid-Structure-Interaction group, where he worked on the design of soft-bodied, bioinspired, aquatic vehicles. Dr. Giorgio-Serchi holds an MSc from the University of Pisa, Italy, in Marine Technologies and a PhD in Fluid Dynamics from the University of Leeds.

View Details

Teresa Berndtsson / Better Images of AI / Letter Word Text Taxonomy / Licenced by CC-BY 4.0.

We’ve collected some of the articles, opinion pieces, videos and resources relating to large language models. Some of these links also cover other generative models. We will periodically update this list to add any further resources of interest.

How they work What are Generative AI models?, Kate Soule, video from IBM Technology. * What is GPT-4 and how does it differ from ChatGPT?, Alex Hern, The Guardian. * What Is ChatGPT Doing … and Why Does It Work?, Stephen Wolfram. * Understanding Large Language Models — A Transformative Reading List, Sebastian Raschka. * How ChatGPT is Trained, video by Ari Seff. * ChatGPT – what is it? How does it work? Should we be excited? Or scared?Deep Dhillon*, The Radical AI podcast.

Journal, conference and arXiv articles Scientists’ Perspectives on the Potential for Generative AI in their Fields, Meredith Ringel Morris, arXiv. * LaMDA: Language Models for Dialog Applications, Romal Thoppilan et al, arXiv. * What Language Model to Train if You Have One Million GPU Hours?, Teven Le Scao et al, arXiv. * Alpaca: A Strong, Replicable Instruction-Following Model, Rohan Taori et al. * Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets, Irene Solaiman, Christy Dennison, NeurIPS 2021. * On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? , Emily Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell, FAccT 2021. * A Survey of Large Language Models, Wayne Xin Zhao et al, arXiv. * A Watermark for Large Language Models, John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein, arXiv. * Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision, Milagros Miceli, Martin Schuessler, Tianling Yang, Proceedings of the ACM on Human-Computer Interaction. * AI classifier for indicating AI-written text, OpenAI. * Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling, Stella Biderman et al, arXiv. * GPT-4 Technical Report, OpenAI, arXiv. * GPT-4 System Card, OpenAI. * BloombergGPT: A Large Language Model for Finance, Shijie Wu et al*, arXiv.

Newspaper, magazine, University website, and blogpost articles Why exams intended for humans might not be good benchmarks for LLMs like GPT-4, Ben Dickson, Venture Beat. * Does GPT-4 Really Understand What We’re Saying?, David Krakauer, Nautilus. * Large language models are biased. Can logic help save them?, Rachel Gordon, MIT News. * Ecosystems graph for ML models and their relationships, researchers at Stanford University. * ChatGPT struggles with Wordle puzzles, which says a lot about how it works, Michael G. Madden, The Conversation. * AIhub coffee corner: Large language models for scientific writing, AIhub. * ChatGPT Is a Blurry JPEG of the Web, Ted Chiang, The New Yorker. * ChatGPT, Galactica, and the Progress Trap, Abeba Birhane and Deborah Raji, Wired. * ChatGPT can’t lie to you, but you still shouldn’t trust it, Mackenzie Graham, The Conversation. * AI information retrieval: A search engine researcher explains the promise and peril of letting ChatGPT and its cousins search the web for you, Chirag Shah, The Conversation. * A small step for research but a giant leap for utility, Interview with Fredrik Heintz, Linköping University. * ChatGPT threatens language diversity. More needs to be done to protect our differences in the age of AI, Collin Bjork, The Conversation. * Column: Afraid of AI? The startups selling it want you to be, Brian Merchant, Los Angeles Times. * Three ways AI chatbots are a security disaster, Melissa Heikkilä, MIT Tech Review. * Time: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic, Billy Perrigo, TIME. * Misplaced fears of an ‘evil’ ChatGPT obscure the real harm being done, John Naughton, The Guardian. * Darktrace warns of rise in AI-enhanced scams since ChatGPT release, Mark Sweney, The Guardian. * Lawmakers struggle to differentiate AI and human emails, Kate Blackwood, Cornell Chronicle. * Colombian judge says he used ChatGPT in ruling, Luke Taylor*, The Guardian. * Bhashini: At your service an Indian language chatbot powered by ChatGPT, video from The Economic Times.

Podcasts and video discussions* The Limitations of ChatGPT with Emily M. Bender and Casey Fiesler, Radical AI Podcast. * CLAIRE AQuA: “ChatGPT and Large Language Models”, CLAIRE. * Su Lin Blodgett on Creating Just Language Technologies, The Good Robot Podcast.

Focus on LLMs and robotics ChatGPT for Robotics: Design Principles and Model Abilities, Microsoft. * OpenAI and Figure join the race to humanoid robot workers, Loz Blain, New Atlas. * Inner Monologue: Embodied Reasoning through Planning with Language Models, Wenlong Huang et al., arXiv. * PaLM-E: An embodied multimodal language model, Danny Driess*, Google. * Consciousness, Embodiment, Language Models (with Professor Murray Shanahan), YouTube video from Machine Learning Street Talk.

Focus on LLMs and education Opinion: ChatGPT – what does it mean for academic integrity?, Giselle Byrnes, Massey University. * Debate: ChatGPT offers unseen opportunities to sharpen students’ critical skills, Erika Darics, Lotte van Poppel, The Conversation. * ChatGPT and cheating: 5 ways to change how students are graded, Louis Volante, Christopher DeLuca Don A. Klinger, The Conversation. * ChatGPT: students could use AI to cheat, but it’s a chance to rethink assessment altogether, Sam Illingworth, The Conversation. * A Teacher’s Prompt Guide to ChatGPT, @herfteducator*.

Relating to art and other creative processes ‘ChatGPT said I did not exist’: how artists and writers are fighting back against AI, Vanessa Thorpe, The Guardian. * AI and the future of work: 5 experts on what ChatGPT, DALL-E and other AI tools mean for artists and knowledge workers, Lynne Parker, Casey Greene, Daniel Acuña, Kentaro Toyama Mark Finlayson, The Conversation. * Is there a way to pay content creators whose work is used to train AI? Yes, but it’s not foolproof, Brendan Paul Murphy, The Conversation. * ChatGPT is the push higher education needs to rethink assessment, Sioux McKenna, Dan Dixon, Daniel Oppenheimer, Margaret Blackie, Sam Illingworth*, The Conversation. * AI Art: How artists are using and confronting machine learning, YouTube video from the Museum of Modern Art.

Misinformation, fake news and the impact on journalism Misinformation Monitor: March 2023, focus on GPT-4, NewsGuard. * A fake news frenzy: why ChatGPT could be disastrous for truth in journalism, Emily Bell, The Guardian. * Defending Against Neural Fake News, Rowan Zellers et al*, arXiv.

Regulation and policy ‘Political propaganda’: China clamps down on access to ChatGPT, Helen Davidson*, The Guardian. * Chatbots, deepfakes, and voice clones: AI deception for sale, USA Federal Trade Commission blog post.

View Details

Emotionally intelligent’ robots could improve their interactions with people. Andriy Onufriyenko/Moment via Getty Images

By Ramana Vinjamuri (Assistant Professor of Computer Science and Electrical Engineering, University of Maryland, Baltimore County)

Robots are machines that can sense the environment and use that information to perform an action. You can find them nearly everywhere in industrialized societies today. There are household robots that vacuum floors and warehouse robots that pack and ship goods. Lab robots test hundreds of clinical samples a day. Education robots support teachers by acting as one-on-one tutors, assistants and discussion facilitators. And medical robotics composed of prosthetic limbs can enable someone to grasp and pick up objects with their thoughts.

Figuring out how humans and robots can collaborate to effectively carry out tasks together is a rapidly growing area of interest to the scientists and engineers that design robots as well as the people who will use them. For successful collaboration between humans and robots, communication is key.

Robotics can help patients recover physical function in rehabilitation. BSIP/Universal Images Group via Getty Images

How people communicate with robotsRobots were originally designed to undertake repetitive and mundane tasks and operate exclusively in robot-only zones like factories. Robots have since advanced to work collaboratively with people with new ways to communicate with each other.

Cooperative control is one way to transmit information and messages between a robot and a person. It involves combining human abilities and decision making with robot speed, accuracy and strength to accomplish a task.

For example, robots in the agriculture industry can help farmers monitor and harvest crops. A human can control a semi-autonomous vineyard sprayer through a user interface, as opposed to manually spraying their crops or broadly spraying the entire field and risking pesticide overuse.

Robots can also support patients in physical therapy. Patients who had a stroke or spinal cord injury can use robots to practice hand grasping and assisted walking during rehabilitation.

Another form of communication, emotional intelligence perception, involves developing robots that adapt their behaviors based on social interactions with humans. In this approach, the robot detects a person’s emotions when collaborating on a task, assesses their satisfaction, then modifies and improves its execution based on this feedback.

For example, if the robot detects that a physical therapy patient is dissatisfied with a specific rehabilitation activity, it could direct the patient to an alternate activity. Facial expression and body gesture recognition ability are important design considerations for this approach. Recent advances in machine learning can help robots decipher emotional body language and better interact with and perceive humans.

Robots in rehabQuestions like how to make robotic limbs feel more natural and capable of more complex functions like typing and playing musical instruments have yet to be answered.

I am an electrical engineer who studies how the brain controls and communicates with other parts of the body, and my lab investigates in particular how the brain and hand coordinate signals between each other. Our goal is to design technologies like prosthetic and wearable robotic exoskeleton devices that could help improve function for individuals with stroke, spinal cord and traumatic brain injuries.

One approach is through brain-computer interfaces, which use brain signals to communicate between robots and humans. By accessing an individual’s brain signals and providing targeted feedback, this technology can potentially improve recovery time in stroke rehabilitation. Brain-computer interfaces may also help restore some communication abilities and physical manipulation of the environment for patients with motor neuron disorders.

Brain-computer interfaces could allow people to control robotic arms by thought alone. Ramana Kumar Vinjamuri, CC BY-ND

The future of human-robot interactionEffective integration of robots into human life requires balancing responsibility between people and robots, and designating clear roles for both in different environments.

As robots are increasingly working hand in hand with people, the ethical questions and challenges they pose cannot be ignored. Concerns surrounding privacy, bias and discrimination, security risks and robot morality need to be seriously investigated in order to create a more comfortable, safer and trustworthy world with robots for everyone. Scientists and engineers studying the “dark side” of human-robot interaction are developing guidelines to identify and prevent negative outcomes.

Human-robot interaction has the potential to affect every aspect of daily life. It is the collective responsibility of both the designers and the users to create a human-robot ecosystem that is safe and satisfactory for all.

Ramana Vinjamuri receives funding from National Science Foundation.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

View Details

Advanced robotics can help surgeons carry out procedures where there is little margin for error. © Microsure BV, 2022

In a surgery in India, a robot scans a patient’s knee to figure out how best to carry out a joint replacement. Meanwhile, in an operating room in the Netherlands, another robot is performing highly challenging microsurgery under the control of a doctor using joysticks.

Such scenarios look set to become more common. At present, some manual operations are so difficult they can be performed by only a small number of surgeons worldwide, while others are invasive and depend on a surgeon’s specific skill.

Advanced robotics are providing tools that have the potential to enable more surgeons to carry out such operations and do so with a higher rate of success.

‘We’re entering the next revolution in medicine,’ said Sophie Cahen, chief executive officer and co-founder of Ganymed Robotics in Paris.

New kneesCahen leads the EU-funded Ganymed project, which is developing a compact robot to make joint-replacement operations more precise, less invasive and – by extension – safer.

The initial focus is on a type of surgery called total knee arthroplasty (TKA), though Ganymed is looking to expand to other joints including the shoulder, ankle and hip.

Ageing populations and lifestyle changes are accelerating demand for such surgery, according to Cahen. Interest in Ganymed’s robot has been expressed in many quarters, including distributors in emerging economies such as India.

‘Demand is super-high because arthroplasty is driven by the age and weight of patients, which is increasing all over the world,’ Cahen said.

Arm with eyesGanymed’s robot will aim to perform two main functions: contactless localisation of bones and collaboration with surgeons to support joint-replacement procedures.

It comprises an arm mounted with ‘eyes’, which use advanced computer-vision-driven intelligence to examine the exact position and orientation of a patient’s anatomical structure. This avoids the need to insert invasive rods and optical trackers into the body.

“We’re entering the next revolution in medicine.”

– Sophie Cahen, Ganymed

Surgeons can then perform operations using tools such as sagittal saws – used for orthopaedic procedures – in collaboration with the robotic arm.

The ‘eyes’ aid precision by providing so-called haptic feedback, which prevents the movement of instruments beyond predefined virtual boundaries. The robot also collects data that it can process in real time and use to hone procedures further.

Ganymed has already carried out a clinical study on 100 patients of the bone-localisation technology and Cahen said it achieved the desired precision.

‘We were extremely pleased with the results – they exceeded our expectations,’ she said.

Now the firm is performing studies on the TKA procedure, with hopes that the robot will be fully available commercially by the end of 2025 and become a mainstream tool used globally.

‘We want to make it affordable and accessible, so as to democratise access to quality care and surgery,’ said Cahen.

Microscopic matters Robots are being explored not only for orthopaedics but also for highly complex surgery at the microscopic level.

The EU-funded MEETMUSA project has been further developing what it describes as the world’s first surgical robot for microsurgery certified under the EU’s ‘CE’ regulatory regime.

Called MUSA, the small, lightweight robot is attached to a platform equipped with arms able to hold and manipulate microsurgical instruments with a high degree of precision. The platform is suspended above the patient during an operation and is controlled by the surgeon through specially adapted joysticks.

In a 2020 study, surgeons reported using MUSA to treat breast-cancer-related lymphedema – a chronic condition that commonly occurs as a side effect of cancer treatment and is characterised by a swelling of body tissues as a result of a build-up of fluids.

MUSA’s robotic arms. Microsure BV, 2022

To carry out the surgery, the robot successfully sutured – or connected – tiny lymph vessels measuring 0.3 to 0.8 millimetre in diameter to nearby veins in the affected area.

‘Lymphatic vessels are below 1 mm in diameter, so it requires a lot of skill to do this,’ said Tom Konert, who leads MEETMUSA and is a clinical field specialist at robot-assisted medical technology company Microsure in Eindhoven, the Netherlands. ‘But with robots, you can more easily do it. So far, with regard to the clinical outcomes, we see really nice results.’

Steady handsWhen such delicate operations are conducted manually, they are affected by slight shaking in the hands, even with highly skilled surgeons, according to Konert. With the robot, this problem can be avoided.

MUSA can also significantly scale down the surgeon’s general hand movements rather than simply repeating them one-to-one, allowing for even greater accuracy than with conventional surgery.

‘When a signal is created with the joystick, we have an algorithm that will filter out the tremor,’ said Konert. ‘It downscales the movement as well. This can be by a factor-10 or 20 difference and gives the surgeon a lot of precision.’

In addition to treating lymphedema, the current version of MUSA – the second, after a previous prototype – has been used for other procedures including nerve repair and soft-tissue reconstruction of the lower leg.

Next generationMicrosure is now developing a third version of the robot, MUSA-3, which Konert expects to become the first one available on a widespread commercial basis.

“When a signal is created with the joystick, we have an algorithm that will filter out the tremor.”

– Tom Konert, MEETMUSA

This new version will have various upgrades, such as better sensors to enhance precision and improved manoeuvrability of the robot’s arms. It will also be mounted on a cart with wheels rather than a fixed table to enable easy transport within and between operating theatres.

Furthermore, the robots will be used with exoscopes – a novel high-definition digital camera system. This will allow the surgeon to view a three-dimensional screen through goggles in order to perform ‘heads-up microsurgery’ rather than the less-comfortable process of looking through a microscope.

Konert is confident that MUSA-3 will be widely used across Europe and the US before a 2029 target date.

‘We are currently finalising product development and preparing for clinical trials of MUSA-3,’ he said. ‘These studies will start in 2024, with approvals and start of commercialisation scheduled for 2025 to 2026.’

MEETMUSA is also looking into the potential of artificial intelligence (AI) to further enhance robots. However, Konert believes that the aim of AI solutions may be to guide surgeons towards their goals and support them in excelling rather than achieving completely autonomous surgery.

‘I think the surgeon will always be there in the feedback loop, but these tools will definitely help the surgeon perform at the highest level in the future,’ he said.

Research in this article was funded via the EU’s European Innovation Council (EIC).

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

Claire chatted to Kat Thiel from Manchester Metropolitan University all about collaborative robots, micro-factories, and fashion manufacturing.

Kat Thiel is a Senior Research Associate at Manchester Metropolitan University’s Manchester Fashion Institute with a research focus on Fashion Practice Research and Industry 4.0., investigating agile cobotic tooling solutions for localised fashion manufacturing. Previously a researcher at the Royal College of Art, she worked on the Future Fashion Factory report ‘Benchmarking the Feasibility of the Micro-Factory Model for the UK Fashion Industry’ and co-produced produced the highly influential report ‘Reshoring UK Garment Manufacturing with Automation’ with Innovate UK KTN.

View Details

Commercial and industrial deployments of robot fleets: package delivery (top left), food delivery (bottom left), e-commerce order fulfillment at Ambi Robotics (top right), autonomous taxis at Waymo (bottom right).

In the last few years we have seen an exciting development in robotics and artificial intelligence: large fleets of robots have left the lab and entered the real world. Waymo, for example, has over 700 self-driving cars operating in Phoenix and San Francisco and is currently expanding to Los Angeles. Other industrial deployments of robot fleets include applications like e-commerce order fulfillment at Amazon and Ambi Robotics as well as food delivery at Nuro and Kiwibot.

Figure 1: “Interactive Fleet Learning” (IFL) refers to robot fleets in industry and academia that fall back on human teleoperators when necessary and continually learn from them over time.

These robots use recent advances in deep learning to operate autonomously in unstructured environments. By pooling data from all robots in the fleet, the entire fleet can efficiently learn from the experience of each individual robot. Furthermore, due to advances in cloud robotics, the fleet can offload data, memory, and computation (e.g., training of large models) to the cloud via the Internet. This approach is known as “Fleet Learning,” a term popularized by Elon Musk in 2016 press releases about Tesla Autopilot and used in press communications by Toyota Research Institute, Wayve AI, and others. A robot fleet is a modern analogue of a fleet of ships, where the word fleet has an etymology tracing back to flēot (‘ship’) and flēotan (‘float’) in Old English.

Data-driven approaches like fleet learning, however, face the problem of the “long tail”: the robots inevitably encounter new scenarios and edge cases that are not represented in the dataset. Naturally, we can’t expect the future to be the same as the past! How, then, can these robotics companies ensure sufficient reliability for their services?

One answer is to fall back on remote humans over the Internet, who can interactively take control and “tele-operate” the system when the robot policy is unreliable during task execution. Teleoperation has a rich history in robotics: the world’s first robots were teleoperated during WWII to handle radioactive materials, and the Telegarden pioneered robot control over the Internet in 1994. With continual learning, the human teleoperation data from these interventions can iteratively improve the robot policy and reduce the robots’ reliance on their human supervisors over time. Rather than a discrete jump to full robot autonomy, this strategy offers a continuous alternative that approaches full autonomy over time while simultaneously enabling reliability in robot systems today.

The use of human teleoperation as a fallback mechanism is increasingly popular in modern robotics companies: Waymo calls it “fleet response,” Zoox calls it “TeleGuidance,” and Amazon calls it “continual learning.” Last year, a software platform for remote driving called Phantom Auto was recognized by Time Magazine as one of their Top 10 Inventions of 2022. And just last month, John Deere acquired SparkAI, a startup that develops software for resolving edge cases with humans in the loop.

A remote human teleoperator at Phantom Auto, a software platform for enabling remote driving over the Internet.

Despite this growing trend in industry, however, there has been comparatively little focus on this topic in academia. As a result, robotics companies have had to rely on ad hoc solutions for determining when their robots should cede control. The closest analogue in academia is interactive imitation learning (IIL), a paradigm in which a robot intermittently cedes control to a human supervisor and learns from these interventions over time. There have been a number of IIL algorithms in recent years for the single-robot, single-human setting including DAgger and variants such as HG-DAgger, SafeDAgger, EnsembleDAgger, and ThriftyDAgger; nevertheless, when and how to switch between robot and human control is still an open problem. This is even less understood when the notion is generalized to robot fleets, with multiple robots and multiple human supervisors.

IFL Formalism and AlgorithmsTo this end, in a recent paper at the Conference on Robot Learning we introduced the paradigm of Interactive Fleet Learning (IFL), the first formalism in the literature for interactive learning with multiple robots and multiple humans. As we’ve seen that this phenomenon already occurs in industry, we can now use the phrase “interactive fleet learning” as unified terminology for robot fleet learning that falls back on human control, rather than keep track of the names of every individual corporate solution (“fleet response”, “TeleGuidance”, etc.). IFL scales up robot learning with four key components:

1. On-demand supervision. Since humans cannot effectively monitor the execution of multiple robots at once and are prone to fatigue, the allocation of robots to humans in IFL is automated by some allocation policy . Supervision is requested “on-demand” by the robots rather than placing the burden of continuous monitoring on the humans.
2. Fleet supervision. On-demand supervision enables effective allocation of limited human attention to large robot fleets. IFL allows the number of robots to significantly exceed the number of humans (e.g., by a factor of 10:1 or more).
3. Continual learning. Each robot in the fleet can learn from its own mistakes as well as the mistakes of the other robots, allowing the amount of required human supervision to taper off over time.
4. The Internet. Thanks to mature and ever-improving Internet technology, the human supervisors do not need to be physically present. Modern computer networks enable real-time remote teleoperation at vast distances.

In the Interactive Fleet Learning (IFL) paradigm, M humans are allocated to the robots that need the most help in a fleet of N robots (where N can be much larger than M). The robots share policy and learn from human interventions over time.

We assume that the robots share a common control policy and that the humans share a common control policy . We also assume that the robots operate in independent environments with identical state and action spaces (but not identical states). Unlike a robot swarm of typically low-cost robots that coordinate to achieve a common objective in a shared environment, a robot fleet simultaneously executes a shared policy in distinct parallel environments (e.g., different bins on an assembly line).

The goal in IFL is to find an optimal supervisor allocation policy , a mapping from (the state of all robots at time t) and the shared policy to a binary matrix that indicates which human will be assigned to which robot at time t. The IFL objective is a novel metric we call the “return on human effort” (ROHE):

where the numerator is the total reward across robots and timesteps and the denominator is the total amount of human actions across robots and timesteps. Intuitively, the ROHE measures the performance of the fleet normalized by the total human supervision required. See the paper for more of the mathematical details.

Using this formalism, we can now instantiate and compare IFL algorithms (i.e., allocation policies) in a principled way. We propose a family of IFL algorithms called Fleet-DAgger, where the policy learning algorithm is interactive imitation learning and each Fleet-DAgger algorithm is parameterized by a unique priority function that each robot in the fleet uses to assign itself a priority score. Similar to scheduling theory, higher priority robots are more likely to receive human attention. Fleet-DAgger is general enough to model a wide range of IFL algorithms, including IFL adaptations of existing single-robot, single-human IIL algorithms such as EnsembleDAgger and ThriftyDAgger. Note, however, that the IFL formalism isn’t limited to Fleet-DAgger: policy learning could be performed with a reinforcement learning algorithm like PPO, for instance.

IFL Benchmark and ExperimentsTo determine how to best allocate limited human attention to large robot fleets, we need to be able to empirically evaluate and compare different IFL algorithms. To this end, we introduce the IFL Benchmark, an open-source Python toolkit available on Github to facilitate the development and standardized evaluation of new IFL algorithms. We extend NVIDIA Isaac Gym, a highly optimized software library for end-to-end GPU-accelerated robot learning released in 2021, without which the simulation of hundreds or thousands of learning robots would be computationally intractable. Using the IFL Benchmark, we run large-scale simulation experiments with N = 100 robots, M = 10 algorithmic humans, 5 IFL algorithms, and 3 high-dimensional continuous control environments (Figure 1, left).

We also evaluate IFL algorithms in a real-world image-based block pushing task with N = 4 robot arms and M = 2 remote human teleoperators (Figure 1, right). The 4 arms belong to 2 bimanual ABB YuMi robots operating simultaneously in 2 separate labs about 1 kilometer apart, and remote humans in a third physical location perform teleoperation through a keyboard interface when requested. Each robot pushes a cube toward a unique goal position randomly sampled in the workspace; the goals are programmatically generated in the robots’ overhead image observations and automatically resampled when the previous goals are reached. Physical experiment results suggest trends that are approximately consistent with those observed in the benchmark environments.

Takeaways and Future DirectionsTo address the gap between the theory and practice of robot fleet learning as well as facilitate future research, we introduce new formalisms, algorithms, and benchmarks for Interactive Fleet Learning. Since IFL does not dictate a specific form or architecture for the shared robot control policy, it can be flexibly synthesized with other promising research directions. For instance, diffusion policies, recently demonstrated to gracefully handle multimodal data, can be used in IFL to allow heterogeneous human supervisor policies. Alternatively, multi-task language-conditioned Transformers like RT-1 and PerAct can be effective “data sponges” that enable the robots in the fleet to perform heterogeneous tasks despite sharing a single policy. The systems aspect of IFL is another compelling research direction: recent developments in cloud and fog robotics enable robot fleets to offload all supervisor allocation, model training, and crowdsourced teleoperation to centralized servers in the cloud with minimal network latency.

While Moravec’s Paradox has so far prevented robotics and embodied AI from fully enjoying the recent spectacular success that Large Language Models (LLMs) like GPT-4 have demonstrated, the “bitter lesson” of LLMs is that supervised learning at unprecedented scale is what ultimately leads to the emergent properties we observe. Since we don’t yet have a supply of robot control data nearly as plentiful as all the text and image data on the Internet, the IFL paradigm offers one path forward for scaling up supervised robot learning and deploying robot fleets reliably in today’s world.

This post is based on the paper “Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision” by Ryan Hoque, Lawrence Chen, Satvik Sharma, Karthik Dharmarajan, Brijen Thananjeyan, Pieter Abbeel, and Ken Goldberg, presented at the Conference on Robot Learning (CoRL) 2022. For more details, see the paper on arXiv, CoRL presentation video on YouTube, open-source codebase on Github, high-level summary on Twitter, and project website.

If you would like to cite this article, please use the following bibtex:

@article{ifl_blog, title={Interactive Fleet Learning}, author={Hoque, Ryan}, url={https://bair.berkeley.edu/blog/2023/04/06/ifl/}, journal={Berkeley Artificial Intelligence Research Blog}, year={2023} }

View Details

MIT researchers developed a soft-rigid robotic finger that incorporates powerful sensors along its entire length, enabling them to produce a robotic hand that could accurately identify objects after only one grasp. Image: Courtesy of the researchers

By Adam Zewe | MIT News Office

Inspired by the human finger, MIT researchers have developed a robotic hand that uses high-resolution touch sensing to accurately identify an object after grasping it just one time.

Many robotic hands pack all their powerful sensors into the fingertips, so an object must be in full contact with those fingertips to be identified, which can take multiple grasps. Other designs use lower-resolution sensors spread along the entire finger, but these don’t capture as much detail, so multiple regrasps are often required.

Instead, the MIT team built a robotic finger with a rigid skeleton encased in a soft outer layer that has multiple high-resolution sensors incorporated under its transparent “skin.” The sensors, which use a camera and LEDs to gather visual information about an object’s shape, provide continuous sensing along the finger’s entire length. Each finger captures rich data on many parts of an object simultaneously.

Using this design, the researchers built a three-fingered robotic hand that could identify objects after only one grasp, with about 85 percent accuracy. The rigid skeleton makes the fingers strong enough to pick up a heavy item, such as a drill, while the soft skin enables them to securely grasp a pliable item, like an empty plastic water bottle, without crushing it.

These soft-rigid fingers could be especially useful in an at-home-care robot designed to interact with an elderly individual. The robot could lift a heavy item off a shelf with the same hand it uses to help the individual take a bath.

“Having both soft and rigid elements is very important in any hand, but so is being able to perform great sensing over a really large area, especially if we want to consider doing very complicated manipulation tasks like what our own hands can do. Our goal with this work was to combine all the things that make our human hands so good into a robotic finger that can do tasks other robotic fingers can’t currently do,” says mechanical engineering graduate student Sandra Liu, co-lead author of a research paper on the robotic finger.

Liu wrote the paper with co-lead author and mechanical engineering undergraduate student Leonardo Zamora Yañez and her advisor, Edward Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research will be presented at the RoboSoft Conference.

A human-inspired fingerThe robotic finger is comprised of a rigid, 3D-printed endoskeleton that is placed in a mold and encased in a transparent silicone “skin.” Making the finger in a mold removes the need for fasteners or adhesives to hold the silicone in place.

The researchers designed the mold with a curved shape so the robotic fingers are slightly curved when at rest, just like human fingers.

“Silicone will wrinkle when it bends, so we thought that if we have the finger molded in this curved position, when you curve it more to grasp an object, you won’t induce as many wrinkles. Wrinkles are good in some ways — they can help the finger slide along surfaces very smoothly and easily — but we didn’t want wrinkles that we couldn’t control,” Liu says.

The endoskeleton of each finger contains a pair of detailed touch sensors, known as GelSight sensors, embedded into the top and middle sections, underneath the transparent skin. The sensors are placed so the range of the cameras overlaps slightly, giving the finger continuous sensing along its entire length.

The GelSight sensor, based on technology pioneered in the Adelson group, is composed of a camera and three colored LEDs. When the finger grasps an object, the camera captures images as the colored LEDs illuminate the skin from the inside.

Image: Courtesy of the researchers

Using the illuminated contours that appear in the soft skin, an algorithm performs backward calculations to map the contours on the grasped object’s surface. The researchers trained a machine-learning model to identify objects using raw camera image data.

As they fine-tuned the finger fabrication process, the researchers ran into several obstacles.

First, silicone has a tendency to peel off surfaces over time. Liu and her collaborators found they could limit this peeling by adding small curves along the hinges between the joints in the endoskeleton.

When the finger bends, the bending of the silicone is distributed along the tiny curves, which reduces stress and prevents peeling. They also added creases to the joints so the silicone is not squashed as much when the finger bends.

While troubleshooting their design, the researchers realized wrinkles in the silicone prevent the skin from ripping.

“The usefulness of the wrinkles was an accidental discovery on our part. When we synthesized them on the surface, we found that they actually made the finger more durable than we expected,” she says.

Getting a good graspOnce they had perfected the design, the researchers built a robotic hand using two fingers arranged in a Y pattern with a third finger as an opposing thumb. The hand captures six images when it grasps an object (two from each finger) and sends those images to a machine-learning algorithm which uses them as inputs to identify the object.

Because the hand has tactile sensing covering all of its fingers, it can gather rich tactile data from a single grasp.

“Although we have a lot of sensing in the fingers, maybe adding a palm with sensing would help it make tactile distinctions even better,” Liu says.

In the future, the researchers also want to improve the hardware to reduce the amount of wear and tear in the silicone over time and add more actuation to the thumb so it can perform a wider variety of tasks.

This work was supported, in part, by the Toyota Research Institute, the Office of Naval Research, and the SINTEF BIFROST project.

View Details

Image source: Bitcraze

Yes, you heard that correctly: the goal is permanent airtime. Robotic flies roaming a room in RoboHouse with no human guidance – achieved within six months. In the future, 24/7 swarms like these may revolutionise aircraft inspection. Imagine a fighter jet enveloped by hundreds of nano drones that build-up a detailed picture in minutes. It’s a challenging mission, but not all challenges are equal. So we asked each Crazyflies team member: What is your favourite problem?

Lennart #myfavouritedesignproblemOkay, maybe permanent flying is exaggerating a bit, at some point batteries need recharging, but it remains the overall design essence. For team member Lennart, this is the main challenge: “We want to optimise the charging process so that you have as many drones in the air as possible with a minimum amount of charging pads.”

Each Crazyflie can buzz off for seven minutes before needing a 35 minute recharge. Through the use of wireless charging pads, human intervention is cancelled out, the alternative being manual battery replacement.

Seppe #myfavouritedesignproblemBut challenges go way further than just battery strategy. Student Seppe identifies his favourite obstacle-to-overcome in collision avoidence: “This does not only include collisions between drones, but also with stationary objects,” Seppe tells us. “By deploying sensors and proper coding, these risks are minimised. Yet the strength of a robust system doesn’t lie in reducing risks, it lies in handling them when they happen.”

Servaas #myfavouritedesignproblemServaas’s favourite challenge ties in with that of his colleague: round-trip latency. Or in English: the time it takes for the flying AI-insects to send their observations and receive commands in return. “Depending on how much time this transfer of information takes up, we could for instance let the drones react to more unpredictable objects such as humans.” Perhaps actual flies could also identify as such an object.

The robotic flies are tested in a drone cage to help further development and reaching their team goals.Andreas #myfavouritedesignproblemFloating away from technical aspects, Andreas defines solving real-world problems his goal: “Designing an autonomous, 24/7 flying drone swarm is cool, but we also want to have an actual impact through real-world application.” Andreas seeks to fulfil this wish by doing market research and identifying problems that yet remain devoid of a solution. One such application could be the inspection of large or difficult-to-access infrastructure like bridges or power lines.

Andrea #myfavouritedesignproblemNot coming from a robotic background, for fifth team member Andrea the challenge amounted to familiarising all this software involved. Luckily, Andrea managed to learn the tools of the trade, finding the AI-insects’ autonomy one of the next exciting challenges to be tackled.

Recently this student team even received the NLF prize for their work, an award by the Dutch Air and Aerospace Foundation.The dronesBut wait, this does not yet complete the team. There are a hundred other individuals, quite literally also team members. The students have included the Crazyflies in their team, deciding to name them ‘member 6 to 105’. These drones are going to inspect infrastructure all by themselves, only stopping occasionally to recharge their batteries.

CyberzooIf all goes well, the Crazyflies could become part of the Crazy Zoo robot exhibition on TU Delft Campus, an initiative by Chris Verhoeven, theme leader swarm robots at TU Delft. For now though, the students have a lot of work on their hands to realise their dreams and live up to the challenges. We have no doubt they will fly high.

The post Robotic flies to swarm 24/7 in RoboHouse appeared first on RoboHouse.

View Details

Nico, Emil, and Moritz founded ReRun with the mission of making powerful visualization tools free and easily accessible for roboticists. Nico and Emil talk about how these powerful tools help debug the complex problem scopes faced by roboticists. Tune in for more.

Nikolaus West
Co-Founder & CEO
Niko is a second-time founder and software engineer with a computer vision background from Stanford. He’s fanatic about bringing great computer vision and robotics products to the physical world.

Emil Ernerfeldt
Co-Founder & CTO
Emil fell in love with coding over 20 years ago and hasn’t looked back since. He’s the creator of egui, an easy-to-use immediate mode GUI in Rust, that we’re using to build Rerun. He brings a strong perspective from the gaming industry, with a focus on great and blazing fast tools.

Links* ReRun * Download mp3 * Subscribe to Robohub using iTunes, RSS, or Spotify * Support us on Patreon

View Details

Researchers created DribbleBot, a system for in-the-wild dribbling on diverse natural terrains including sand, gravel, mud, and snow using onboard sensing and computing. In addition to these football feats, such robots may someday aid humans in search-and-rescue missions. Photo: Mike Grimmett/MIT CSAIL

By Rachel Gordon | MIT CSAIL

If you’ve ever played soccer with a robot, it’s a familiar feeling. Sun glistens down on your face as the smell of grass permeates the air. You look around. A four-legged robot is hustling toward you, dribbling with determination.

While the bot doesn’t display a Lionel Messi-like level of ability, it’s an impressive in-the-wild dribbling system nonetheless. Researchers from MIT’s Improbable Artificial Intelligence Lab, part of the Computer Science and Artificial Intelligence Laboratory (CSAIL), have developed a legged robotic system that can dribble a soccer ball under the same conditions as humans. The bot used a mixture of onboard sensing and computing to traverse different natural terrains such as sand, gravel, mud, and snow, and adapt to their varied impact on the ball’s motion. Like every committed athlete, “DribbleBot” could get up and recover the ball after falling.

Programming robots to play soccer has been an active research area for some time. However, the team wanted to automatically learn how to actuate the legs during dribbling, to enable the discovery of hard-to-script skills for responding to diverse terrains like snow, gravel, sand, grass, and pavement. Enter, simulation.

A robot, ball, and terrain are inside the simulation — a digital twin of the natural world. You can load in the bot and other assets and set physics parameters, and then it handles the forward simulation of the dynamics from there. Four thousand versions of the robot are simulated in parallel in real time, enabling data collection 4,000 times faster than using just one robot. That’s a lot of data.

Video: MIT CSAIL

The robot starts without knowing how to dribble the ball — it just receives a reward when it does, or negative reinforcement when it messes up. So, it’s essentially trying to figure out what sequence of forces it should apply with its legs. “One aspect of this reinforcement learning approach is that we must design a good reward to facilitate the robot learning a successful dribbling behavior,” says MIT PhD student Gabe Margolis, who co-led the work along with Yandong Ji, research assistant in the Improbable AI Lab. “Once we’ve designed that reward, then it’s practice time for the robot: In real time, it’s a couple of days, and in the simulator, hundreds of days. Over time it learns to get better and better at manipulating the soccer ball to match the desired velocity.”

The bot could also navigate unfamiliar terrains and recover from falls due to a recovery controller the team built into its system. This controller lets the robot get back up after a fall and switch back to its dribbling controller to continue pursuing the ball, helping it handle out-of-distribution disruptions and terrains.

“If you look around today, most robots are wheeled. But imagine that there’s a disaster scenario, flooding, or an earthquake, and we want robots to aid humans in the search-and-rescue process. We need the machines to go over terrains that aren’t flat, and wheeled robots can’t traverse those landscapes,” says Pulkit Agrawal, MIT professor, CSAIL principal investigator, and director of Improbable AI Lab.” The whole point of studying legged robots is to go terrains outside the reach of current robotic systems,” he adds. “Our goal in developing algorithms for legged robots is to provide autonomy in challenging and complex terrains that are currently beyond the reach of robotic systems.”

The fascination with robot quadrupeds and soccer runs deep — Canadian professor Alan Mackworth first noted the idea in a paper entitled “On Seeing Robots,” presented at VI-92, 1992. Japanese researchers later organized a workshop on “Grand Challenges in Artificial Intelligence,” which led to discussions about using soccer to promote science and technology. The project was launched as the Robot J-League a year later, and global fervor quickly ensued. Shortly after that, “RoboCup” was born.

Compared to walking alone, dribbling a soccer ball imposes more constraints on DribbleBot’s motion and what terrains it can traverse. The robot must adapt its locomotion to apply forces to the ball to dribble. The interaction between the ball and the landscape could be different than the interaction between the robot and the landscape, such as thick grass or pavement. For example, a soccer ball will experience a drag force on grass that is not present on pavement, and an incline will apply an acceleration force, changing the ball’s typical path. However, the bot’s ability to traverse different terrains is often less affected by these differences in dynamics — as long as it doesn’t slip — so the soccer test can be sensitive to variations in terrain that locomotion alone isn’t.

“Past approaches simplify the dribbling problem, making a modeling assumption of flat, hard ground. The motion is also designed to be more static; the robot isn’t trying to run and manipulate the ball simultaneously,” says Ji. “That’s where more difficult dynamics enter the control problem. We tackled this by extending recent advances that have enabled better outdoor locomotion into this compound task which combines aspects of locomotion and dexterous manipulation together.”

On the hardware side, the robot has a set of sensors that let it perceive the environment, allowing it to feel where it is, “understand” its position, and “see” some of its surroundings. It has a set of actuators that lets it apply forces and move itself and objects. In between the sensors and actuators sits the computer, or “brain,” tasked with converting sensor data into actions, which it will apply through the motors. When the robot is running on snow, it doesn’t see the snow but can feel it through its motor sensors. But soccer is a trickier feat than walking — so the team leveraged cameras on the robot’s head and body for a new sensory modality of vision, in addition to the new motor skill. And then — we dribble.

“Our robot can go in the wild because it carries all its sensors, cameras, and compute on board. That required some innovations in terms of getting the whole controller to fit onto this onboard compute,” says Margolis. “That’s one area where learning helps because we can run a lightweight neural network and train it to process noisy sensor data observed by the moving robot. This is in stark contrast with most robots today: Typically a robot arm is mounted on a fixed base and sits on a workbench with a giant computer plugged right into it. Neither the computer nor the sensors are in the robotic arm! So, the whole thing is weighty, hard to move around.”

There’s still a long way to go in making these robots as agile as their counterparts in nature, and some terrains were challenging for DribbleBot. Currently, the controller is not trained in simulated environments that include slopes or stairs. The robot isn’t perceiving the geometry of the terrain; it’s only estimating its material contact properties, like friction. If there’s a step up, for example, the robot will get stuck — it won’t be able to lift the ball over the step, an area the team wants to explore in the future. The researchers are also excited to apply lessons learned during development of DribbleBot to other tasks that involve combined locomotion and object manipulation, quickly transporting diverse objects from place to place using the legs or arms.

The research is supported by the DARPA Machine Common Sense Program, the MIT-IBM Watson AI Lab, the National Science Foundation Institute of Artificial Intelligence and Fundamental Interactions, the U.S. Air Force Research Laboratory, and the U.S. Air Force Artificial Intelligence Accelerator. The paper will be presented at the 2023 IEEE International Conference on Robotics and Automation (ICRA).

View Details

The robotic system is shown in an experimental hive © Artificial Life Lab/U. of Graz/Hiveopolis

By Celia Luterbacher

Honeybees are famously finicky when it comes to being studied. Research instruments and conditions and even unfamiliar smells can disrupt a colony’s behavior. Now, a joint research team from the Mobile Robotic Systems Group in EPFL’s School of Engineering and School of Computer and Communication Sciences and the Hiveopolis project at Austria’s University of Graz have developed a robotic system that can be unobtrusively built into the frame of a standard honeybee hive.

Composed of an array of thermal sensors and actuators, the system measures and modulates honeybee behavior through localized temperature variations.

“Many rules of bee society – from collective and individual interactions to raising a healthy brood – are regulated by temperature, so we leveraged that for this study,” explains EPFL PhD student Rafael Barmak, first author on a paper on the system recently published in Science Robotics. “The thermal sensors create a snapshot of the bees’ collective behavior, while the actuators allow us to influence their movement by modulating thermal fields.”

“Previous studies on the thermal behavior of honeybees in winter have relied on observing the bees or manipulating the outside temperature,” adds Martin Stefanec of the University of Graz. “Our robotic system enables us to change the temperature from within the cluster, emulating the heating behavior of core bees there, and allowing us to study how the winter cluster actively regulates its temperature.”

A ‘biohybrid superorganism’ to mitigate colony collapseBee colonies are challenging to study in winter since they are sensitive to cold, and opening their hives risks harming them in addition to influencing their behavior. But thanks to the researchers’ biocompatible robotic system, they were able to study three experimental hives, located at the Artificial Life Lab at the University of Graz, during winter and to control them remotely from EPFL. Inside the device, a central processor coordinated the sensors, sent commands to the actuators, and transmitted data to the scientists, demonstrating that the system could be used to study bees with no intrusion – or even cameras – required.

Mobile Robotic Systems Group head Francesco Mondada explains that one of the most important aspects of the system – which he calls a ‘biohybrid superorganism’ for its combination of robotics with a colony of individuals acting as a living entity – is its ability to simultaneously observe and influence bee behavior.

“By gathering data on the bees’ position and creating warmer areas in the hive, we were able to encourage them to move around in ways they would never normally do in nature during the winter, when they tend to huddle together to conserve energy. This gives us the possibility to act on behalf of a colony, for example by directing it toward a food source, or discouraging it from dividing into too-small groups, which can threaten its survival.”

The robotic system is shown in an experimental hive © MOBOTS / EPFL / Hiveopolis

The scientists were able to prolong the survival of a colony following the death of its queen by distributing heat energy via the actuators. The system’s ability to mitigate colony collapse could have implications for bee survivability, which has become a growing environmental and food security concern as the pollinators’ global populations have declined.

Never-before-seen behaviorsIn addition to its potential to support colonies, the system has shed light on honeybee behaviors that have never been observed, opening new avenues in biological research.

“The local thermal stimuli produced by our system revealed previously unreported dynamics that are generating exciting new questions and hypotheses,” says EPFL postdoctoral researcher and corresponding author Rob Mills. “For example, currently, no model can explain why we were able to encourage the bees to cross some cold temperature ‘valleys’ within the hive.”

The researchers now plan to use the system to study bees in summertime, which is a critical period for raising young. In parallel, the Mobile Robotic Systems Group is exploring systems using vibrational pathways to interact with honeybees.

“The biological acceptance aspect of this work is critical: the fact that the bees accepted the integration of electronics into the hive gives our device great potential for different scientific or agricultural applications,” says Mondada.

This work was supported by the EU H2020 FET project HIVEOPOLIS (no. 824069), coordinated by Thomas Schmickl, and by the Field of Excellence COLIBRI (Complexity of Life in basic Research and Innovation) at the University of Graz.

• PAPER – A robotic honeycomb for interaction with a honeybee colony. R. Barmak and M. Stefanec, D. N. Hofstadler, L. Piotet, S. Schönwetter-Fuchs-Schistek, F. Mondada, T. Schmickl, and R. Mills. Science Robotics, vol 8, Issue 76. https://doi.org/10.1126/scirobotics.add7385

View Details

Claire chatted to Maitreyee Wairagkar from the University of California all about neurotechnology, artificial intelligence, and assistive robotics.

Maitreyee Wairagkar is a postdoctoral fellow at University of California, Davis, developing assistive neurotechnology using artificial intelligence to restore lost function in people with neurological disorders. She builds brain-computer interfaces to enable people with severe motor and speech impairments to communicate directly via their brain signals by breaking barriers between humans and technology. Previously, she was at Imperial College London and UK Dementia Research Institute where she developed conversational AI and social robots for dementia support.

View Details

Robotics and AI are poised to fundamentally change the future of healthcare. © Elnur, Shutterstock

In a Swiss classroom, two children are engrossed in navigating an intricate maze with the help of a small, rather cute, robot. The interaction is easy and playful – it is also providing researchers with valuable information on how children learn and the conditions in which information is most effectively absorbed.

Rapid improvements in intuitive human-machine interactions (HMI) are poised to kick off big changes in society. In particular, two European research projects give a sense of how these trends could influence two core areas: education and healthcare.

Child learningIn EU-funded ANIMATAS, a cross-border network of universities and industrial partners is exploring if, and how, robots and artificial intelligence (AI) can help us learn more effectively. One idea is around making mistakes: children can learn by spotting and correcting others’ errors – and having a robot make them might be useful.

‘A teacher can’t make mistakes,’ said project coordinator Professor Mohamed Chetouani of the Sorbonne University in Paris, France. ‘But a robot? They could. And mistakes are very useful in education.’

According to Prof Chetouani, it is simplistic to ask questions like ‘can robots help children learn better’ because learning is such a complex concept. He said that, for example, any automatic assumption that pupils who concentrate on lessons are learning more isn’t necessarily true.

That’s why, from the start, the project set out to ask smarter, more specific questions that would help identify just how robots could be useful in classrooms.

ANIMATAS is made up of sub-projects each led by an early-stage researcher. One of the sub-project goals was to better understand the learning process in children and analyse what types of interaction best help them to retain information.

“Mistakes are very useful in education.”

– Professor Mohamed Chetouani, ANIMATAS

Robot rolesAn experiment set up to investigate this question invited children to team up with the aptly named QTRobot to find the most efficient route around a map.

During the exercise, the robot reacts interactively with the children to offer tips and suggestions. It is also carefully measuring various indicators in the children’s body language such as eye contact and direction, tone of voice and facial expression.

As hoped, researchers did indeed find that certain patterns of interaction corresponded with improved learning. With this information, they will be better able to evaluate how well children are engaging with educational material and, in the longer term, develop strategies to maximise such engagement – thereby boosting learning potential.

Future steps will include looking at how to adapt this robot-enhanced learning to children with special educational needs.

‘We believe that it could be really important in this context,’ said Prof Chetouani.

Help at handAki Härmä, a researcher at Philips Research Eindhoven in the Netherlands, believes that robotics and AI are going to fundamentally change healthcare.

“Healthcare can be 24/7.”

Aki Härmä, PhilHumans

In the EU-funded PhilHumans project that he is coordinating, early-stage researchers from five universities across Europe work with two commercial partners – R2M Solution in Spain and Philips Electronics in the Netherlands – to learn how innovative technologies can improve people’s health.

AI makes new services possible and ‘it means healthcare can be 24/7,’ Härmä said.

He points to the vast potential for technology to help people manage their own health from home: apps able to track a person’s mental and physical state and spot problems early on, chatbots that can give advice and propose diagnoses, and algorithms for robots to navigate safely around abodes.

Empathetic botsThe project, which started in 2019 and will run until late 2023, is made of up of eight sub-projects, each led by a doctoral student.

One sub-project, supervised by Phillips researcher Rim Helaoui, is looking at how the specific skills of mental-health practitioners – such as empathy and open-ended questioning – may be encoded into an AI-powered chatbot. This could mean that people with mental-health conditions would be able to access relevant support from home, potentially at a lower cost.

The team quickly realised that replicating the full range of psychotherapeutic skills in a chatbot would involve challenges that could not be solved all at once. It focused instead on one key challenge: how to generate a bot that displayed empathy.

‘This is the essential first step to get people to feel they can open up and share,’ said Helaoui.

As a starting point, the team produced an algorithm able to respond with the appropriate tone and content to convey empathy. The technology has yet to be converted into an app or product, but provides a building block that could be used in many different applications.

Rapid advancesPhilHumans is also exploring other possibilities for the application of AI in healthcare. An algorithm is being developed that can use ‘camera vision’ to understand the tasks that a person is trying to carry out and analyse the surrounding environment.

The ultimate goal would be to use this algorithm in a home-assistant robot to help people with cognitive decline complete everyday tasks successfully.

One thing that has helped the project overall, said Härmä, is the speed with which other organisations have been developing natural language processors with impressive capabilities, like GPT-3 from OpenAI. The project expects to be able to harness the unexpectedly rapid improvements in these and other areas to advance faster.

Both ANIMATAS and PhilHumans are actively working on expanding the limits of intuitive HMI.

In doing so, they have provided a valuable training ground for young researchers and given them important exposure to the commercial world. Overall, the two projects are ensuring that a new generation of highly skilled researchers is equipped to lead the way forward in HMI and its potential applications.

Research in this article was funded via the EU’s Marie Skłodowska-Curie Actions (MSCA).

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

The automotive industry has the largest number of robots working in factories around the world: Operational stock hit a new record of about one million units. This represents about one third of the total number installed across all industries.

“The automotive industry effectively invented automated manufacturing,” says Marina Bill, President of the International Federation of Robotics. “Today, robots are playing a vital role in enabling this industry’s transition from combustion engines to electric power. Robotic automation helps car manufacturers manage the wholesale changes to long-established manufacturing methods and technologies.”

Robot density in automotiveRobot density is a key indicator which illustrates the current level of automation in the top car producing economies: In the Republic of Korea, 2,867 industrial robots per 10,000 employees were in operation in 2021. Germany ranks in second place with 1,500 units followed by the United States counting 1,457 units and Japan with 1,422 units per 10,000 workers.

The world´s biggest car manufacturer, China, has a robot density of 772 units, but is catching up fast: Within a year, new robot installations in the Chinese automotive industry almost doubled to 61,598 units in 2021- accounting for 52% of the total 119,405 units installed in factories around the world.

Electric vehicles drive automationAmbitious political targets for electric vehicles are forcing the car industry to invest: The European Union has announced plans to end the sale of air-polluting vehicles by 2035. The US government aims to reach a voluntary goal of 50% market share for electric vehicle sales by 2030 and all new vehicles sold in China must be powered by “new energy” by 2035. Half of them must be electric, fuel cell, or plug-in hybrid – the remaining 50%, hybrid vehicles.

Most automotive manufacturers who have already invested in traditional “caged” industrial robots for basic assembling are now also investing in collaborative applications for final assembly and finishing tasks. Tier-two automotive parts suppliers, many of which are SMEs, are slower to automate fully. Yet, as robots become smaller, more adaptable, easier to program, and less capital-intensive this is expected to change.

View Details

Claire chatted to Thom Kirwan-Evans from Origami Labs all about computer vision, machine learning, and robots in industry.

Thom Kirwan-Evans is a co-founder at Origami Labs where he applies the latest AI research to solve complex real world problems. Thom started as a physicist at Dstl working with camera systems before moving to an engineering consultancy and then setting up his own company last year. A keen runner and father of two, a key aim in starting his business was a good work-life balance.

View Details

MIT researchers have developed resilient artificial muscles that can enable insect-scale aerial robots to effectively recover flight performance after suffering severe damage. Photo: Courtesy of the researchers

By Adam Zewe | MIT News Office

Bumblebees are clumsy fliers. It is estimated that a foraging bee bumps into a flower about once per second, which damages its wings over time. Yet despite having many tiny rips or holes in their wings, bumblebees can still fly.

Aerial robots, on the other hand, are not so resilient. Poke holes in the robot’s wing motors or chop off part of its propellor, and odds are pretty good it will be grounded.

Inspired by the hardiness of bumblebees, MIT researchers have developed repair techniques that enable a bug-sized aerial robot to sustain severe damage to the actuators, or artificial muscles, that power its wings — but to still fly effectively.

They optimized these artificial muscles so the robot can better isolate defects and overcome minor damage, like tiny holes in the actuator. In addition, they demonstrated a novel laser repair method that can help the robot recover from severe damage, such as a fire that scorches the device.

Using their techniques, a damaged robot could maintain flight-level performance after one of its artificial muscles was jabbed by 10 needles, and the actuator was still able to operate after a large hole was burnt into it. Their repair methods enabled a robot to keep flying even after the researchers cut off 20 percent of its wing tip.

This could make swarms of tiny robots better able to perform tasks in tough environments, like conducting a search mission through a collapsing building or dense forest.

“We spent a lot of time understanding the dynamics of soft, artificial muscles and, through both a new fabrication method and a new understanding, we can show a level of resilience to damage that is comparable to insects,” says Kevin Chen, the D. Reid Weedon, Jr. Assistant Professor in the Department of Electrical Engineering and Computer Science (EECS), the head of the Soft and Micro Robotics Laboratory in the Research Laboratory of Electronics (RLE), and the senior author of the paper on these latest advances. “We’re very excited about this. But the insects are still superior to us, in the sense that they can lose up to 40 percent of their wing and still fly. We still have some catch-up work to do.”

Chen wrote the paper with co-lead authors Suhan Kim and Yi-Hsuan Hsiao, who are EECS graduate students; Younghoon Lee, a postdoc; Weikun “Spencer” Zhu, a graduate student in the Department of Chemical Engineering; Zhijian Ren, an EECS graduate student; and Farnaz Niroui, the EE Landsman Career Development Assistant Professor of EECS at MIT and a member of the RLE. The article appeared in Science Robotics.

Robot repair techniquesUsing the repair techniques developed by MIT researchers, this microrobot can still maintain flight-level performance even after the artificial muscles that power its wings were jabbed by 10 needles and 20 percent of one wing tip was cut off. Credit: Courtesy of the researchers.

The tiny, rectangular robots being developed in Chen’s lab are about the same size and shape as a microcassette tape, though one robot weighs barely more than a paper clip. Wings on each corner are powered by dielectric elastomer actuators (DEAs), which are soft artificial muscles that use mechanical forces to rapidly flap the wings. These artificial muscles are made from layers of elastomer that are sandwiched between two razor-thin electrodes and then rolled into a squishy tube. When voltage is applied to the DEA, the electrodes squeeze the elastomer, which flaps the wing.

But microscopic imperfections can cause sparks that burn the elastomer and cause the device to fail. About 15 years ago, researchers found they could prevent DEA failures from one tiny defect using a physical phenomenon known as self-clearing. In this process, applying high voltage to the DEA disconnects the local electrode around a small defect, isolating that failure from the rest of the electrode so the artificial muscle still works.

Chen and his collaborators employed this self-clearing process in their robot repair techniques.

First, they optimized the concentration of carbon nanotubes that comprise the electrodes in the DEA. Carbon nanotubes are super-strong but extremely tiny rolls of carbon. Having fewer carbon nanotubes in the electrode improves self-clearing, since it reaches higher temperatures and burns away more easily. But this also reduces the actuator’s power density.

“At a certain point, you will not be able to get enough energy out of the system, but we need a lot of energy and power to fly the robot. We had to find the optimal point between these two constraints — optimize the self-clearing property under the constraint that we still want the robot to fly,” Chen says.

However, even an optimized DEA will fail if it suffers from severe damage, like a large hole that lets too much air into the device.

Chen and his team used a laser to overcome major defects. They carefully cut along the outer contours of a large defect with a laser, which causes minor damage around the perimeter. Then, they can use self-clearing to burn off the slightly damaged electrode, isolating the larger defect.

“In a way, we are trying to do surgery on muscles. But if we don’t use enough power, then we can’t do enough damage to isolate the defect. On the other hand, if we use too much power, the laser will cause severe damage to the actuator that won’t be clearable,” Chen says.

The team soon realized that, when “operating” on such tiny devices, it is very difficult to observe the electrode to see if they had successfully isolated a defect. Drawing on previous work, they incorporated electroluminescent particles into the actuator. Now, if they see light shining, they know that part of the actuator is operational, but dark patches mean they successfully isolated those areas.

The new research could make swarms of tiny robots better able to perform tasks in tough environments, like conducting a search mission through a collapsing building or dense forest. Photo: Courtesy of the researchers

Flight test successOnce they had perfected their techniques, the researchers conducted tests with damaged actuators — some had been jabbed by many needles while other had holes burned into them. They measured how well the robot performed in flapping wing, take-off, and hovering experiments.

Even with damaged DEAs, the repair techniques enabled the robot to maintain its flight performance, with altitude, position, and attitude errors that deviated only very slightly from those of an undamaged robot. With laser surgery, a DEA that would have been broken beyond repair was able to recover 87 percent of its performance.

“I have to hand it to my two students, who did a lot of hard work when they were flying the robot. Flying the robot by itself is very hard, not to mention now that we are intentionally damaging it,” Chen says.

These repair techniques make the tiny robots much more robust, so Chen and his team are now working on teaching them new functions, like landing on flowers or flying in a swarm. They are also developing new control algorithms so the robots can fly better, teaching the robots to control their yaw angle so they can keep a constant heading, and enabling the robots to carry a tiny circuit, with the longer-term goal of carrying its own power source.

“This work is important because small flying robots — and flying insects! — are constantly colliding with their environment. Small gusts of wind can be huge problems for small insects and robots. Thus, we need methods to increase their resilience if we ever hope to be able to use robots like this in natural environments,” says Nick Gravish, an associate professor in the Department of Mechanical and Aerospace Engineering at the University of California at San Diego, who was not involved with this research. “This paper demonstrates how soft actuation and body mechanics can adapt to damage and I think is an impressive step forward.”

This work is funded, in part, by the National Science Foundation (NSF) and a MathWorks Fellowship.

• PAPER – Laser-assisted failure recovery for dielectric elastomer actuators in aerial robots. Suhan Kim, Yi-Hsuan Hsiao, Younghoon Lee, Weikun Zhu, Zhijian Ren, Farnaz Niroui, and Yufeng Chen. Science Robotics, 8(76), eadf4278.

View Details

Source: Unsplash

According to statistics, the healthcare drone industry has witnessed a dramatic surge in the last couple of years. In 2020, the market grew 30% and is expected to grow from $254 million in 2021 to $1,5 billion in 2028. The most common use case for healthcare drones is the delivery of medical supplies and laboratory samples.

However, it appears that in 2022, new ways of using drones have become available. Research groups in the USA have completed test drone organ delivery operations and have done so successfully. How will the proliferation of organ transportation with drones influence the healthcare industry?

What is an organ transportation drone?Before we talk about how medical delivery drones may influence the healthcare industry, it’s worth investigating what they are and how they work.

Drones are unmanned aerial vehicles (UAVs) that can be operated remotely or that can fly autonomously using on-board sensors and GPS. The smallest drones can be as small as 30sm in length and weigh about 500 grams. The largest can reach the size of a track and carry weights up to 4.5 tons.

Drones for organ transportation are somewhere in the middle. Organs are usually delivered in batches and can actually be quite heavy with all the ecosystem that is necessary to maintain them in the desired condition. Organ delivery drones are able to carry freight up to 180kg. These drones are designed to transport vital organs such as hearts, kidneys, and livers from one location to another in a safe and efficient manner.

Drones are able to transport objects on relatively short distances. While iner-city and inter-city delivery is possible it’s probably too early to talk about international transportation. This limitation can be explained by the difficulties of piloting the drone as well as by the nature of organ transplantation as such that is an extremely timely matter.

Why are organ transportation drones so important?Right now organ delivery drones are still on the stage of development and testing. However, a survey conducted among surgeons in the USA has shown that this innovation may have high importance for the field.

A survey by University of Maryland Medical Center in Baltimore has shown that 76.4% of organ transplantation surgeons believe that cold ischaemia time reduction to 8 hours, achieved via the use of organ delivery drones, would increase organ acceptance rates. In fact, time to delivery reduction is one of the most significant benefits of using drones for the delivery of organs. After the organs have been extracted from the body, they only have 4 to 72 hours to be transplanted. The longer the waiting time the higher are the chances of the organ failing upon transplantation. Only 16% of surgeons believed the current transportation system is adequate for organ delivery needs. Clara Guerrero, director of communications for the Texas Organ Sharing Alliance, says in the article for San Antonio Report, ‘You’re saving hours. What that also means is the organ is more viable. That person, they don’t have to wait so long for the organ to arrive. We’re saving lives faster and sooner’.

Another research has investigated the potential drawbacks and benefits of using organ transportation drones as opposed to delivery with commercial aircrafts and charter flights. They have used a modified, six-rotor UAS to model organ delivery. During the transportation process, they’ve measured the temperature and vibration levels. This is what they write:

“Temperatures remained stable and low (2.5 °C). Pressure changes (0.37–0.86 kPa) correlated with increased altitude. Drone travel was associated with less vibration (<0.5 G) than was observed with fixed-wing flight (>2.0 G). Peak velocity was 67.6 km/h (42 m/h). Biopsies of the kidney taken prior to and after organ shipment revealed no damage resulting from drone travel. The longest flight was 3.0 miles, modeling an organ flight between two inner city hospitals.”

— Joseph R. Scalea et al, University of Maryland

In the future, the use of drones for organ transportation could greatly increase as the technology improves. For example, advances in autonomous flight systems and improved battery technology could make it possible for drones to fly longer distances and reach more remote locations. Additionally, the development of drone delivery networks could make it possible to deliver organs to hospitals and other healthcare facilities in a matter of minutes, reducing the time that vital organs are outside of a human body.

Who makes drones for organ transportation?Currently, there are several companies in the world that are working on making organ transplantation a reality.

One such company is Zipline, based in California, USA. The company has developed a drone specifically for the transportation of medical supplies, including blood and organs. The drone is able to fly at high speeds and cover long distances, making it ideal for transporting organs between hospitals and other medical facilities.

Another company, Matternet, is also based in California and it has developed a similar drone for medical deliveries. This drone is used to deliver diagnostic samples in Switzerland and can be applied for carrying small organs as well.

A Canadian company Unither Bioélectronique specializes in quick and efficient delivery methods for organ transportation such as drones. The Indian government is developing an organ delivery drone system together with.

In China, a company called EHang has developed a drone that can transport organs and other medical supplies. This drone is able to fly at high speeds and cover long distances, making it ideal for transporting organs between hospitals and other medical facilities.

In Europe, a company called Volocopter, based in Germany, has developed a drone specifically for the transportation of organs. The drone is equipped with advanced navigation systems and can fly at high speeds, making it ideal for transporting organs between hospitals and other medical facilities.

In India, the first human organ delivery drone developed by MGM Healthcare. It can be used to transport organs with a maximum distance of 20 kilometers.

ConclusionThe use of organ drone delivery represents a significant breakthrough in the field of organ transplantation. This innovative technology has the potential to revolutionize the way organs are transported, making the process faster, more efficient, and more reliable than ever before. By reducing the time it takes to deliver organs to transplant centers, drones could help save countless lives by ensuring that patients receive the organs they need in a timely manner. Moreover, by reducing the risk of organ damage during transport, drones could improve the success rates of organ transplants, leading to better outcomes for patients. With the ongoing development of organ drone delivery technology, we can look forward to a future where organ transplantation is more accessible, reliable, and effective than ever before.

View Details

Robotics and AI can help build healthier bee colonies, benefitting biodiversity and food supply. © 0 Lorenzo Bernini 0, Shutterstock.com

The robotic bee replicants home in on the unsuspecting queen of a hive. But unlike the rebellious replicants in the 1982 sci-fi thriller Blade Runner, these ones are here to work.

Combining miniature robotics, artificial intelligence (AI) and machine learning, the plan is for the robotic bees to stimulate egg laying in the queen by, for example, feeding her the right foods at the right time.

Survive and thrive‘We plan to affect a whole ecosystem by interacting with only one single animal, the queen,’ said Dr Farshad Arvin, a roboticist and computer scientist at the University of Durham in the UK. ‘If we can keep activities like egg laying happening at the right time, we are expecting to have healthier broods and more active and healthy colonies. This will then improve pollination.’

While that goes on above the surface, shape-morphing robot roots that can adapt and interact with real plants and fungi are hard at work underground. There, plants and their fungal partners form vast networks.

These robotic bees and roots are being developed by two EU-funded projects. Both initiatives are looking into how artificial versions of living things central to maintaining ecosystems can help real-life organisms and their environment survive and thrive – while ensuring food for people remains plentiful.

“If we can keep activities like egg laying happening at the right time, we are expecting to have healthier broods.”

– Dr Farshad Arvin, RoboRoyale

That could be crucial to the planet’s long-term future, particularly with many species currently facing steep population declines as a result of threats that include habitat loss, pollution and climate change.

One of those at risk is the honeybee, a keystone species in the insect pollination required for 75% of crops grown for human food globally.

Fit for a queenThe RoboRoyale project that Arvin leads combines microrobotic, biological and machine-learning technologies to nurture the queen honeybee’s well-being. The project is funded by the European Innovation Council’s Pathfinder programme.

A unique aspect of RoboRoyale is its sole focus on the queen rather than the entire colony, according to Arvin. He said the idea is to demonstrate how supporting a single key organism can stimulate production in the whole environment, potentially affecting hundreds of millions of organisms.

The multi-robot system, which the team hopes to start testing in the coming months, will learn over time how to groom the queen to optimise her egg laying and production of pheromones – chemical scents that influence the behaviour of the hive.

The system is being deployed in artificial glass observation hives in Austria and Turkey, with the bee replicants designed to replace the so-called court bees that normally interact with the queen.

Foods for broodsOne aim is that the robot bees can potentially stimulate egg laying by providing the queen with specific protein-rich foods at just the right time to boost this activity. In turn, an expected benefit is that a resulting increase in bees and foraging flights would mean stronger pollination of the surrounding ecosystem to support plant growth and animals.

The system enables six to eight robotic court bees, some equipped with microcameras, to be steered inside an observation hive by a controller attached to them from outside. The end goal is to make the robot bees fully autonomous.

The concept design of RoboRoyale robotic controller. © Farshad Arvin, 2023

Prior to this, the RoboRoyale team observed queen bees in several hives using high-resolution cameras and image-analysis software to get more insight into their behaviour.

The team captured more than 150 million samples of the queens’ trajectories inside the hive and detailed footage of their social interactions with other bees. It is now analysing the data.

Once the full robotic system is sufficiently tested, the RoboRoyale researchers hope it will foster understanding of the potential for bio-hybrid technology not only in bees but also in other organisms.

‘It might lead to a novel type of sustainable technology that positively impacts surrounding ecosystems,’ said Arvin.

Wood Wide WebThe other project, I-Wood, is exploring a very different type of social network – one that’s underground.

Scientists at the Italian Institute of Technology (IIT) in Genoa are studying what they call the Wood Wide Web. It consists of plant roots connected to each other through a symbiotic network of fungi that provide them with nutrients and help them to share resources and communicate.

“Biomimicry in robotics and technology will have a fundamental role in saving our planet.”

– Dr Barbara Mazzolai, I-Wood

To understand these networks better and find ways to stimulate their growth, I-Wood is developing soft, shape-changing robotic roots that can adapt and interact with real plants and fungi. The idea is for a robotic plant root to use a miniaturised 3D printer in its tip to enable it to grow and branch out, layer by layer, in response to environmental factors such as temperature, humidity and available nutrients.

‘These technologies will help to increase knowledge about the relationship between symbionts and hosts,’ said Dr Barbara Mazzolai, an IIT roboticist who leads the project.

Mazzolai’s team has a greenhouse where it grows rice plants inoculated with fungi. So far, the researchers have separately examined the growth of roots and fungi.

Soon, they plan to merge their findings to see how, when and where the interaction between the two occurs and what molecules it involves.

The findings can later be used by I-Wood’s robots to help the natural symbiosis between fungi and roots work as effectively as possible. The team hopes to start experimenting with robots in the greenhouse by the end of this year.

The robotic roots can be programmed to move autonomously, helped by sensors in their tips, according to Mazzolai. Like the way real roots or earthworms move underground, they will also seek passages that are easier to move through due to softer or less compact soil.

Tweaks of the tradeBut there are challenges in combining robotics with nature.

For example, bees are sensitive to alien objects in their hive and may remove them or coat them in wax. This makes it tricky to use items like tracking tags.

The bees have, however, become more accepting after the team tweaked elements of the tags such as their coating, materials and smell, according to Arvin of RoboRoyale.

Despite these challenges, Arvin and Mazzolai believe robotics and artificial intelligence could play a key part in sustaining ecosystems and the environment in the long term. For Mazzolai, the appeal lies in the technologies’ potential to offer deeper analysis of little-understood interactions among plants, animals and the environment.

For instance, with the underground web of plant roots and fungi believed to be crucial to maintaining healthy ecosystems and limiting global warming by locking up carbon, the project’s robotic roots can help shed light on how we can protect and support these natural processes.

‘Biomimicry in robotics and technology will have a fundamental role in saving our planet,’ Mazzolai said.

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

Claire chatted to Alessandra Rossi from the University of Naples all about social robotics, theory of mind, and robots playing football.

Alessandra Rossi is Assistant Professor at the University of Naples Federico II in Italy. Her PhD thesis was part of the Marie Sklodowska-Curie ETN SECURE project at the University of Hertfordshire in the UK, and she is now a Visiting Lecturer and Researcher there. Her research interests include human-robot interaction, social robotics, explainable AI, multi-agent systems and user profiling. She is the team leader of RoboCup team Bold Hearts at the University of Hertfordshire, and Executive Committee member of the RoboCup Humanoid League.

View Details

A team of MIT engineers is designing a kit of universal robotic parts that an astronaut could easily mix and match to build different robot “species” to fit various missions on the moon. Credit: hexapod image courtesy of the researchers, edited by MIT News

By Jennifer Chu | MIT News Office

When astronauts begin to build a permanent base on the moon, as NASA plans to do in the coming years, they’ll need help. Robots could potentially do the heavy lifting by laying cables, deploying solar panels, erecting communications towers, and building habitats. But if each robot is designed for a specific action or task, a moon base could become overrun by a zoo of machines, each with its own unique parts and protocols.

To avoid a bottleneck of bots, a team of MIT engineers is designing a kit of universal robotic parts that an astronaut could easily mix and match to rapidly configure different robot “species” to fit various missions on the moon. Once a mission is completed, a robot can be disassembled and its parts used to configure a new robot to meet a different task.

The team calls the system WORMS, for the Walking Oligomeric Robotic Mobility System. The system’s parts include worm-inspired robotic limbs that an astronaut can easily snap onto a base, and that work together as a walking robot. Depending on the mission, parts can be configured to build, for instance, large “pack” bots capable of carrying heavy solar panels up a hill. The same parts could be reconfigured into six-legged spider bots that can be lowered into a lava tube to drill for frozen water.

“You could imagine a shed on the moon with shelves of worms,” says team leader George Lordos, a PhD candidate and graduate instructor in MIT’s Department of Aeronautics and Astronautics (AeroAstro), in reference to the independent, articulated robots that carry their own motors, sensors, computer, and battery. “Astronauts could go into the shed, pick the worms they need, along with the right shoes, body, sensors and tools, and they could snap everything together, then disassemble it to make a new one. The design is flexible, sustainable, and cost-effective.”

Lordos’ team has built and demonstrated a six-legged WORMS robot. Last week, they presented their results at IEEE’s Aerospace Conference, where they also received the conference’s Best Paper Award.

MIT team members include Michael J. Brown, Kir Latyshev, Aileen Liao, Sharmi Shah, Cesar Meza, Brooke Bensche, Cynthia Cao, Yang Chen, Alex S. Miller, Aditya Mehrotra, Jacob Rodriguez, Anna Mokkapati, Tomas Cantu, Katherina Sapozhnikov, Jessica Rutledge, David Trumper, Sangbae Kim, Olivier de Weck, Jeffrey Hoffman, along with Aleks Siemenn, Cormac O’Neill, Diego Rivero, Fiona Lin, Hanfei Cui, Isabella Golemme, John Zhang, Jolie Bercow, Prajwal Mahesh, Stephanie Howe, and Zeyad Al Awwad, as well as Chiara Rissola of Carnegie Mellon University and Wendell Chun of the University of Denver.

Animal instinctsWORMS was conceived in 2022 as an answer to NASA’s Breakthrough, Innovative and Game-changing (BIG) Idea Challenge — an annual competition for university students to design, develop, and demonstrate a game-changing idea. In 2022, NASA challenged students to develop robotic systems that can move across extreme terrain, without the use of wheels.

A team from MIT’s Space Resources Workshop took up the challenge, aiming specifically for a lunar robot design that could navigate the extreme terrain of the moon’s South Pole — a landscape that is marked by thick, fluffy dust; steep, rocky slopes; and deep lava tubes. The environment also hosts “permanently shadowed” regions that could contain frozen water, which, if accessible, would be essential for sustaining astronauts.

As they mulled over ways to navigate the moon’s polar terrain, the students took inspiration from animals. In their initial brainstorming, they noted certain animals could conceptually be suited to certain missions: A spider could drop down and explore a lava tube, a line of elephants could carry heavy equipment while supporting each other down a steep slope, and a goat, tethered to an ox, could help lead the larger animal up the side of a hill as it transports an array of solar panels.

“As we were thinking of these animal inspirations, we realized that one of the simplest animals, the worm, makes similar movements as an arm, or a leg, or a backbone, or a tail,” says deputy team leader and AeroAstro graduate student Michael Brown. “And then the lightbulb went off: We could build all these animal-inspired robots using worm-like appendages.’”

The research team in Killian Court at MIT. Credit: Courtesy of the researchers

Snap on, snap offLordos, who is of Greek descent, helped coin WORMS, and chose the letter “O” to stand for “oligomeric,” which in Greek signifies “a few parts.”

“Our idea was that, with just a few parts, combined in different ways, you could mix and match and get all these different robots,” says AeroAstro undergraduate Brooke Bensche.

The system’s main parts include the appendage, or worm, which can be attached to a body, or chassis, via a “universal interface block” that snaps the two parts together through a twist-and-lock mechanism. The parts can be disconnected with a small tool that releases the block’s spring-loaded pins.

Appendages and bodies can also snap into accessories such as a “shoe,” which the team engineered in the shape of a wok, and a LiDAR system that can map the surroundings to help a robot navigate.

“In future iterations we hope to add more snap-on sensors and tools, such as winches, balance sensors, and drills,” says AeroAstro undergraduate Jacob Rodriguez.

The team developed software that can be tailored to coordinate multiple appendages. As a proof of concept, the team built a six-legged robot about the size of a go-cart. In the lab, they showed that once assembled, the robot’s independent limbs worked to walk over level ground. The team also showed that they could quickly assemble and disassemble the robot in the field, on a desert site in California.

In its first generation, each WORMS appendage measures about 1 meter long and weighs about 20 pounds. In the moon’s gravity, which is about one-sixth that of Earth’s, each limb would weigh about 3 pounds, which an astronaut could easily handle to build or disassemble a robot in the field. The team has planned out the specs for a larger generation with longer and slightly heavier appendages. These bigger parts could be snapped together to build “pack” bots, capable of transporting heavy payloads.

“There are many buzz words that are used to describe effective systems for future space exploration: modular, reconfigurable, adaptable, flexible, cross-cutting, et cetera,” says Kevin Kempton, an engineer at NASA’s Langley Research Center, who served as a judge for the 2022 BIG Idea Challenge. “The MIT WORMS concept incorporates all these qualities and more.”

This research was supported, in part, by NASA, MIT, the Massachusetts Space Grant, the National Science Foundation, and the Fannie and John Hertz Foundation.

• PAPER – WORMS: Field-Reconfigurable Robots for Extreme Lunar Terrain. George Lordos, Michael J. Brown, Kir Latyshev, and Aileen Liao. Proceedings of IEEE Aerospace 2023.

View Details

Shua Cho works on her artwork in “Introduction to Physical Computing for Artists” at the MIT Student Art Association. Photo: Sarah Bastille

By Ken Shulman | Arts at MIT

One student confesses that motors have always freaked them out. Amy Huynh, a first-year student in the MIT Technology and Policy Program, says “I just didn’t respond to the way electrical engineering and coding is usually taught.”

Huynh and her fellow students found a different way to master coding and circuits during the Independent Activities Period course Introduction to Physical Computing for Artists — a class created by Student Art Association (SAA) instructor Timothy Lee and offered for the first time last January. During the four-week course, students learned to use circuits, wiring, motors, sensors, and displays by developing their own kinetic artworks.

“It’s a different approach to learning about art, and about circuits,” says Lee, who joined the SAA instructional staff last June after completing his MFA at Goldsmiths, University of London. “Some classes can push the technology too quickly. Here we try to take away the obstacles to learning, to create a collaborative environment, and to frame the technology in the broader concept of making an artwork. For many students, it’s a very effective way to learn.”

Lee graduated from Wesleyan University with three concurrent majors in neuroscience, biology, and studio art. “I didn’t have a lot of free time,” says Lee, who originally intended to attend medical school before deciding to follow his passion for making art. “But I benefited from studying both science and art. Just as I almost always benefited from learning from my peers. I draw on both of those experiences in designing and teaching this class.”

On this January evening, the third of four scheduled classes, Lee leads his students through an exercise to create an MVP — a minimum viable product of their art project. The MVP, he explains, serves as an artist’s proof of concept. “This is the smallest single unit that can demonstrate that your project is doable,” he says. “That you have the bare-minimum functioning hardware and software that shows your project can be scalable to your vision. Our work here is different from pure robotics or pure electronics. Here, the technology and the coding don’t need to be perfect. They need to support your aesthetic and conceptual goals. And here, these things can also be fun.”

Lee distributes various electronic items to the students according to their specific needs — wires, soldering irons, resistors, servo motors, and Arduino components. The students have already acquired a working knowledge of coding and the Arduino language in the first two class sessions. Sophomore Shua Cho is designing an evening gown bedecked with flowers that will open and close continuously. Her MVP is a cluster of three blossoms, mounted on a single post that, when raised and lowered, opens and closes the sewn blossoms. She asks Lee for help in attaching a servo motor — an electronic motor that alternates between 0, 90, and 180 degrees — to the post. Two other students, working on similar problems, immediately pull their chairs beside Cho and Lee to join the discussion.

Shua Cho is designing an evening gown bedecked with flowers that will open and close continuously. Her minimum viable product is a cluster of three blossoms, mounted on a single post that, when raised and lowered, opens and closes the sewn blossoms. Photo: Sarah Bastille

The instructor suggests they observe the dynamics of an old-fashioned train locomotive wheel. One student calls up the image on their laptop. Then, as a group, they reach a solution for Cho — an assembly of wire and glue that will attach the servo engine to the central post, opening and closing the blossoms. It’s improvised, even inelegant. But it works, and proves that the project for the blossom-covered kinetic dress is viable.

“This is one of the things I love about MIT,” says aeronautical and astronautical engineering senior Hannah Munguia. Her project is a pair of hands that, when triggered by a motion sensor, will applaud when anyone walks by. “People raise their hand when they don’t understand something. And other people come to help. The students here trust each other, and are willing to collaborate.”

Student Hannah Munguia (left), instructor Timothy Lee (center), and student Bryan Medina work on artwork in “Introduction to Physical Computing for Artists” at the MIT Student Art Association. Photo: Sarah Bastille

Cho, who enjoys exploring the intersection between fashion and engineering, discovered Lee’s work on Instagram long before she decided to enroll at MIT. “And now I have the chance to study with him,” says Cho, who works at Infinite — MIT’s fashion magazine — and takes classes in both mechanical engineering and design. “I find that having a creative project like this one, with a goal in mind, is the best way for me to learn. I feel like it reinforces my neural pathways, and I know it helps me retain information. I find myself walking down the street or in my room, thinking about possible solutions for this gown. It never feels like work.”

For Lee, who studied computational art during his master’s program, his course is already a successful experiment. He’d like to offer a full-length version of “Introduction to Physical Computing for Artists” during the school year. With 10 sessions instead of four, he says, students would be able to complete their projects, instead of stopping at an MVP.

“Prior to coming to MIT, I’d only taught at art institutions,” says Lee. “Here, I needed to revise my focus, to redefine the value of art education for students who most likely were not going to pursue art as a profession. For me, the new definition was selecting a group of skills that are necessary in making this type of art, but that can also be applied to other areas and fields. Skills like sensitivity to materials, tactile dexterity, and abstract thinking. Why not learn these skills in an atmosphere that is experimental, visually based, sometimes a little uncomfortable. And why not learn that you don’t need to be an artist to make art. You just have to be excited about it.”

View Details

Claire chatted to Edward Timpson from QinetiQ all about robots in the military, uncrewed vehicles, and cyber security.

Ed Timpson joined QinetiQ in 2020 after 11 years serving in the Royal Navy as a Weapons Engineering Officer (Submarines) across a number of ranks. Joining QinetiQ Target Systems as a Project Engineer and also managing the Hardware team, he had success in developing new capabilities for the Banshee family of UAS. He then moved into future systems within QinetiQ as a Principal Systems Engineer specialising in complex trials and experimentation of uncrewed vehicles. He now heads up the Robotics and Autonomous Systems capability within QinetiQ UK.

View Details

Author: D.Farina. Credits: Istituto Italiano di Tecnologia – © IIT, all rights reserved

Researchers at Istituto Italiano di Tecnologia (IIT-Italian Institute of Technology) in Genova has realized a new soft robot inspired by the biology of earthworms,which is able to crawl thanks to soft actuators that elongate or squeeze, when air passes through them or is drawn out. The prototype has been described in the international journal Scientific Reports of the Nature Portfolio, and it is the starting point for developing devices for underground exploration, but also search and rescue operations in confined spaces and the exploration of other planets.

Nature offers many examples of animals, such as snakes, earthworms, snails, and caterpillars, which use both the flexibility of their bodies and the ability to generate physical travelling waves along the length of their body to move and explore different environments. Some of their movements are also similar to plant roots.

Taking inspiration from nature and, at the same time, revealing new biological phenomena while developing new technologies is the main goal of the BioInspired Soft robotics lab coordinated by Barbara Mazzolai, and this earthworm-like robot is the latest invention coming from her group.

The creation of earthworm-like robot was made possible through a thorough understanding and application of earthworm locomotion mechanics. They use alternating contractions of muscle layers to propel themselves both below and above the soil surface by generating retrograde peristaltic waves. The individual segments of their body (metameres) have a specific quantity of fluid that controls the internal pressure to exert forces, and perform independent, localized and variable movement patterns.

IIT researchers have studied the morphology of earthworms and have found a way to mimic their muscle movements, their constant volume coelomic chambers and the function of their bristle-like hairs (setae) by creating soft robotic solutions.

The team developed a peristaltic soft actuator (PSA) that implements the antagonistic muscle movements of earthworms; from a neutral position it elongates when air is pumped into it and compresses when air is extracted from it. The entire body of the robotic earthworm is made of five PSA modules in series, connected with interlinks. The current prototype is 45 cm long and weighs 605 grams.

Each actuator has an elastomeric skin that encapsulates a known amount of fluid, thus mimicking the constant volume of internal coelomic fluid in earthworms. The earthworm segment becomes shorter longitudinally and wider circumferentially and exerts radial forces as the longitudinal muscles of an individual constant volume chamber contract. Antagonistically, the segment becomes longer along the anterior–posterior axis and thinner circumferentially with the contraction of circumferential muscles, resulting in penetration forces along the axis.

Every single actuator demonstrates a maximum elongation of 10.97mm at 1 bar of positive pressure and a maximum compression of 11.13mm at 0.5 bar of negative pressure, unique in its ability to generate both longitudinal and radial forces in a single actuator module.

In order to propel the robot on a planar surface, small passive friction pads inspired by earthworms’ setae were attached to the ventral surface of the robot. The robot demonstrated improved locomotion with a speed of 1.35mm/s.

This study not only proposes a new method for developing a peristaltic earthworm-like soft robot but also provides a deeper understanding of locomotion from a bioinspired perspective in different environments. The potential applications for this technology are vast, including underground exploration, excavation, search and rescue operations in subterranean environments and the exploration of other planets. This bioinspired burrowing soft robot is a significant step forward in the field of soft robotics and opens the door for further advancements in the future.

• PAPER – An earthworm‑like modular soft robot for locomotion in multi‑terrain environments. Riddhi Das, Saravana Prashanth Murali Babu, Francesco Visentin, Stefano Palagi, and Barbara Mazzolai. Sci Rep 13, 1571 (2023). https://doi.org/10.1038/s41598-023-28873-w

View Details

We’ve all seen or heard of the Hype Cycle. It’s a visual depiction of the lifecycle stages a technology goes through from the initial development to commercial maturity. It’s a useful way to track what technologies are compatible with your organization’s needs. There are five stages of the Hype Cycle, which take us through the initial excitement trigger, that leads to the peak of inflated expectations followed by disillusionment. It’s only as a product moves into more tangible market use, sometimes called ‘The Slope of Enlightenment’, that we start to reach full commercial viability.

Working with so many robotics startups, I see this stage as the transition into revenue generation in more than pilot use cases. This is the point where a startup no longer needs to nurture each customer deployment but can produce reference use cases and start to reliably scale. I think this is a useful model but that Gartner’s classifications don’t do robotics justice.

For example, this recent Gartner chart puts Smart Robots at the top of the hype cycle. Robotics is a very fast moving field at the moment. The majority of new robotics companies are less than 5-10 years old. From the perspective of the end user, it can be very difficult to know when a company is moving out of the hype cycle and into commercial maturity because there aren’t many deployments or much marketing at first, particularly compared to the media coverage of companies at the peak of the hype cycle.

So, here’s where I think robotics technologies really fit on the Gartner Hype Cycle:

Innovation trigger

• Voice interfaces for practical applications of robots
• Foundational models applied to robotics

Peak of inflated expectations

• Large Language models – although likely to progress very quickly
• Humanoids

Trough of disillusionment

• Quadrupeds
• Cobots
• Full self-driving cars and trucks
• Powered clothing/Exoskeletons

Slope of enlightenment

• Teleoperation
• Cloud fleet management
• Drones for critical delivery to remote locations
• Drones for civilian surveillance
• Waste recycling
• Warehouse robotics (pick and place)
• Hospital logistics
• Education robots
• Food preparation
• Rehabilitation
• AMRs in other industries

Plateau of productivity

• Robot vacuum cleaners (domestic and commercial)
• Surgical Robots
• Warehouse robotics (AMRs in particular)
• Factory automation (robot arms)
• 3d printing
• ROS
• Simulation

AI, in the form of Large Language Models ie. ChatGPT, GPT3 and Bard is at peak hype, as are humanoid robots, and perhaps the peak of that hype is the idea of RoboGPT, or using LLMs to interpret human commands to robots. Just in the last year, four or five new humanoid robot companies have come out of stealth from Figure, Teslabot, Aeolus, Giant AI, Agility, Halodi, and so far only Halodi has a commercial deployment doing internal security augmentation for ADT.

Cobots are still in the Trough of Disillusionment, in spite of Universal Robot selling 50,000+ arms. People buy robot arms from companies like Universal primarily for affordability, ease of setup, not requiring safety guarding hardware and capable of industrial precision. The full promise of collaborative robots has had trouble landing with end users. We don’t really deploy collaborative robots engaged in frequent hand-offs to humans. Perhaps we need more dual armed cobots with better human-robot interaction before we really explore the possibilities.

Interestingly the Trough of Disillusionment generates a lot of media coverage but it’s usually negative. Self-driving cars and trucks are definitely at the bottom of the trough. Whereas powered clothing or exoskeletons, or quadrupeds are a little harder to place.

AMRs, or Autonomous Mobile Robots, are a form of self-driving cargo that is much more successful than self-driving cars or trucks traveling on public roads. AMRs are primarily deployed in warehouses, hospitals, factories, farms, retail facilities, airports and even on the sidewalk. Behind every successful robot deployment there is probably a cloud fleet management provider or a teleoperation provider, or monitoring service.

Finally, the Plateau of Productivity is where the world’s most popular robots are. Peak popularity is the Roomba and other home robot vacuum cleaners. Before their acquisition by Amazon, iRobot had sold more than 40 million Roombas and captured 20% of the domestic vacuum cleaner market. Now commercial cleaning fleets are switching to autonomy as well.

And of course Productivity (not Hype) is also where the workhorse industrial robot arms live with ever increasing deployments worldwide. The International Federation of Robotics, IFR, reports that more than half a million new industrial robot arms were deployed in 2021, up 31% from 2020. This figure has been rising pretty steadily since I first started tracking robotics back in 2010.

What does your robotics hype cycle look like? What technology would you like me to add to this chart? Contact andra@svrobo.org

View Details

Claire chatted to Dr Maria Bauza Villalonga from DeepMind all about robot learning, transferable skills, and general AI.

Maria Bauza Villalonga is a research scientist at DeepMind. In 2022, she earned her PhD in Robotics at the Massachusetts Institute of Technology, working with Prof. Alberto Rodriguez. Her research focuses on achieving precise robotic generalization by learning probabilistic models of the world that allow robots to reuse their skills across multiple tasks with high success. Maria has received several fellowships including Facebook, NVIDIA, and LaCaixa.

View Details

The final episode of our RoboHouse Interview Trilogy: ‘The Working Life of the Robotics Engineer’ interviews Srimannarayana Baratam. Sriman, as he is also called, co-founded the company Perciv.ai just two months after graduating. Rens van Poppel explores his journey so far.

Perciv.ai claims that AI-driven machine perception could become affordable to everyone. When was this vision formed, and how did it come about? Sriman points to the period right after his graduation. He says it was pivotal for building trust with partners, and consensus with effective communication. Because starting your own company comes with a lot of challenges.

Srimannarayana Baratam was the first to graduate from the MSc Robotics at the TU Delft. The Master’s degree programme was newly launched in 2020 and aims to train students who can guide the industry towards a kind of robotisation that promotes and reinforces workplace attractiveness.“It is important for to find partners you can trust,” says Sriman. “You need to understand each other’s motivation and commitment. You need to assess what real value does this person add to the team.”

Coming from an automotive background in India, Sriman’s master’s thesis investigated the use of radar and cameras to protect vulnerable people in urban environments. He co-founded the start-up with his supervisor, Dr András Pálffy, and Balazs Szekeres, another robotics student who heard about the project. In the two months after his graduation, Sriman and his co-founders came together to focus full-time on their vision for Perciv.ai.

“In July and August we sat down and discussed the vision between the three of us,” he says. This period also led to tough conversations, ranging from finance to market strategy. “When finally the main questions were sorted out, you just got to take that leap of faith together. This leap of faith proved fruitful, seeing that the high level of trust resulted in a high level of productivity over the past five months.”

Since then Perciv.ai went on to win the NWO take-off phase 1 grant, got their own office and workspace in RoboHouse, signed a contract with an unmanned aerial vehicle (UAV) company and in doing so, generated their first sales revenue.

“This does not mean that there are no more heavy debates,” Sriman adds. “We all share the same vision, but in order to reach our goal of a sustainable and affordable product, we sometimes have different ideas on what that final product should look like.”

Sriman’s passion for robotics and the company’s goals is palpable: “We want to make machine perception technology available to all.”

The post RoboHouse Interview Trilogy, Part III: Srimannarayana Baratam and Perciv.ai appeared first on RoboHouse.

View Details

Claire chatted to Dr Jonathan Aitken from the University of Sheffield all about manufacturing, sewer inspection, and robots in the real world.

Jonathan Aitken is a Senior University Teacher in Robotics at the University of Sheffield. His research is focused on building useful, useable, and expandable architectures for future robotics systems. Most recently this has involved building complex digital twins for collaborative robots in manufacturing processes and investigating localisation for robots operating in sewer pipes. His teaching focuses on providing students with the tools to bring distributed computing to complex robotic processes.

View Details

MIT engineers are hoping to help doctors tailor treatments to patients’ specific heart form and function, with a custom robotic heart. The team has developed a procedure to 3D print a soft and flexible replica of a patient’s heart. Image: Melanie Gonick, MIT

By Jennifer Chu | MIT News Office

No two hearts beat alike. The size and shape of the the heart can vary from one person to the next. These differences can be particularly pronounced for people living with heart disease, as their hearts and major vessels work harder to overcome any compromised function.

MIT engineers are hoping to help doctors tailor treatments to patients’ specific heart form and function, with a custom robotic heart. The team has developed a procedure to 3D print a soft and flexible replica of a patient’s heart. They can then control the replica’s action to mimic that patient’s blood-pumping ability.

The procedure involves first converting medical images of a patient’s heart into a three-dimensional computer model, which the researchers can then 3D print using a polymer-based ink. The result is a soft, flexible shell in the exact shape of the patient’s own heart. The team can also use this approach to print a patient’s aorta — the major artery that carries blood out of the heart to the rest of the body.

To mimic the heart’s pumping action, the team has fabricated sleeves similar to blood pressure cuffs that wrap around a printed heart and aorta. The underside of each sleeve resembles precisely patterned bubble wrap. When the sleeve is connected to a pneumatic system, researchers can tune the outflowing air to rhythmically inflate the sleeve’s bubbles and contract the heart, mimicking its pumping action.

The researchers can also inflate a separate sleeve surrounding a printed aorta to constrict the vessel. This constriction, they say, can be tuned to mimic aortic stenosis — a condition in which the aortic valve narrows, causing the heart to work harder to force blood through the body.

Doctors commonly treat aortic stenosis by surgically implanting a synthetic valve designed to widen the aorta’s natural valve. In the future, the team says that doctors could potentially use their new procedure to first print a patient’s heart and aorta, then implant a variety of valves into the printed model to see which design results in the best function and fit for that particular patient. The heart replicas could also be used by research labs and the medical device industry as realistic platforms for testing therapies for various types of heart disease.

“All hearts are different,” says Luca Rosalia, a graduate student in the MIT-Harvard Program in Health Sciences and Technology. “There are massive variations, especially when patients are sick. The advantage of our system is that we can recreate not just the form of a patient’s heart, but also its function in both physiology and disease.”

Rosalia and his colleagues report their results in a study appearing in Science Robotics. MIT co-authors include Caglar Ozturk, Debkalpa Goswami, Jean Bonnemain, Sophie Wang, and Ellen Roche, along with Benjamin Bonner of Massachusetts General Hospital, James Weaver of Harvard University, and Christopher Nguyen, Rishi Puri, and Samir Kapadia at the Cleveland Clinic in Ohio.

Print and pumpIn January 2020, team members, led by mechanical engineering professor Ellen Roche, developed a “biorobotic hybrid heart” — a general replica of a heart, made from synthetic muscle containing small, inflatable cylinders, which they could control to mimic the contractions of a real beating heart.

Shortly after those efforts, the Covid-19 pandemic forced Roche’s lab, along with most others on campus, to temporarily close. Undeterred, Rosalia continued tweaking the heart-pumping design at home.

“I recreated the whole system in my dorm room that March,” Rosalia recalls.

Months later, the lab reopened, and the team continued where it left off, working to improve the control of the heart-pumping sleeve, which they tested in animal and computational models. They then expanded their approach to develop sleeves and heart replicas that are specific to individual patients. For this, they turned to 3D printing.

“There is a lot of interest in the medical field in using 3D printing technology to accurately recreate patient anatomy for use in preprocedural planning and training,” notes Wang, who is a vascular surgery resident at Beth Israel Deaconess Medical Center in Boston.

An inclusive designIn the new study, the team took advantage of 3D printing to produce custom replicas of actual patients’ hearts. They used a polymer-based ink that, once printed and cured, can squeeze and stretch, similarly to a real beating heart.

As their source material, the researchers used medical scans of 15 patients diagnosed with aortic stenosis. The team converted each patient’s images into a three-dimensional computer model of the patient’s left ventricle (the main pumping chamber of the heart) and aorta. They fed this model into a 3D printer to generate a soft, anatomically accurate shell of both the ventricle and vessel.

The action of the soft, robotic models can be controlled to mimic the patient’s blood-pumping ability. Image: Melanie Gonick, MIT

The team also fabricated sleeves to wrap around the printed forms. They tailored each sleeve’s pockets such that, when wrapped around their respective forms and connected to a small air pumping system, the sleeves could be tuned separately to realistically contract and constrict the printed models.

The researchers showed that for each model heart, they could accurately recreate the same heart-pumping pressures and flows that were previously measured in each respective patient.

“Being able to match the patients’ flows and pressures was very encouraging,” Roche says. “We’re not only printing the heart’s anatomy, but also replicating its mechanics and physiology. That’s the part that we get excited about.”

Going a step further, the team aimed to replicate some of the interventions that a handful of the patients underwent, to see whether the printed heart and vessel responded in the same way. Some patients had received valve implants designed to widen the aorta. Roche and her colleagues implanted similar valves in the printed aortas modeled after each patient. When they activated the printed heart to pump, they observed that the implanted valve produced similarly improved flows as in actual patients following their surgical implants.

Finally, the team used an actuated printed heart to compare implants of different sizes, to see which would result in the best fit and flow — something they envision clinicians could potentially do for their patients in the future.

“Patients would get their imaging done, which they do anyway, and we would use that to make this system, ideally within the day,” says co-author Nguyen. “Once it’s up and running, clinicians could test different valve types and sizes and see which works best, then use that to implant.”

Ultimately, Roche says the patient-specific replicas could help develop and identify ideal treatments for individuals with unique and challenging cardiac geometries.

“Designing inclusively for a large range of anatomies, and testing interventions across this range, may increase the addressable target population for minimally invasive procedures,” Roche says.

This research was supported, in part, by the National Science Foundation, the National Institutes of Health, and the National Heart Lung Blood Institute.

• PAPER – Soft robotic patient-specific hydrodynamic model of aortic stenosis and ventricular remodeling. Luca Rosalia, Caglar Ozturk, Debkalpa Goswami, Jean Bonnemain, Sophie X. Wang, Benjamin Bonner, James C. Weaver, Rishi Puri, Samir Kapadia, Christopher T. Nguyen, and Ellen T. Roche. Science Robotics, 8 (75).

View Details

By Jędrzej Orbik, Charles Sun, Coline Devin, Glen Berseth

Reinforcement learning provides a conceptual framework for autonomous agents to learn from experience, analogously to how one might train a pet with treats. But practical applications of reinforcement learning are often far from natural: instead of using RL to learn through trial and error by actually attempting the desired task, typical RL applications use a separate (usually simulated) training phase. For example, AlphaGo did not learn to play Go by competing against thousands of humans, but rather by playing against itself in simulation. While this kind of simulated training is appealing for games where the rules are perfectly known, applying this to real world domains such as robotics can require a range of complex approaches, such as the use of simulated data, or instrumenting real-world environments in various ways to make training feasible under laboratory conditions. Can we instead devise reinforcement learning systems for robots that allow them to learn directly “on-the-job”, while performing the task that they are required to do? In this blog post, we will discuss ReLMM, a system that we developed that learns to clean up a room directly with a real robot via continual learning.

We evaluate our method on different tasks that range in difficulty. The top-left task has uniform white blobs to pickup with no obstacles, while other rooms have objects of diverse shapes and colors, obstacles that increase navigation difficulty and obscure the objects and patterned rugs that make it difficult to see the objects against the ground.

To enable “on-the-job” training in the real world, the difficulty of collecting more experience is prohibitive. If we can make training in the real world easier, by making the data gathering process more autonomous without requiring human monitoring or intervention, we can further benefit from the simplicity of agents that learn from experience. In this work, we design an “on-the-job” mobile robot training system for cleaning by learning to grasp objects throughout different rooms.

Lesson 1: The Benefits of Modular Policies for Robots.People are not born one day and performing job interviews the next. There are many levels of tasks people learn before they apply for a job as we start with the easier ones and build on them. In ReLMM, we make use of this concept by allowing robots to train common-reusable skills, such as grasping, by first encouraging the robot to prioritize training these skills before learning later skills, such as navigation. Learning in this fashion has two advantages for robotics. The first advantage is that when an agent focuses on learning a skill, it is more efficient at collecting data around the local state distribution for that skill.

That is shown in the figure above, where we evaluated the amount of prioritized grasping experience needed to result in efficient mobile manipulation training. The second advantage to a multi-level learning approach is that we can inspect the models trained for different tasks and ask them questions, such as, “can you grasp anything right now” which is helpful for navigation training that we describe next.

Training this multi-level policy was not only more efficient than learning both skills at the same time but it allowed for the grasping controller to inform the navigation policy. Having a model that estimates the uncertainty in its grasp success (Ours above) can be used to improve navigation exploration by skipping areas without graspable objects, in contrast to No Uncertainty Bonus which does not use this information. The model can also be used to relabel data during training so that in the unlucky case when the grasping model was unsuccessful trying to grasp an object within its reach, the grasping policy can still provide some signal by indicating that an object was there but the grasping policy has not yet learned how to grasp it. Moreover, learning modular models has engineering benefits. Modular training allows for reusing skills that are easier to learn and can enable building intelligent systems one piece at a time. This is beneficial for many reasons, including safety evaluation and understanding.

Lesson 2: Learning systems beat hand-coded systems, given time

Many robotics tasks that we see today can be solved to varying levels of success using hand-engineered controllers. For our room cleaning task, we designed a hand-engineered controller that locates objects using image clustering and turns towards the nearest detected object at each step. This expertly designed controller performs very well on the visually salient balled socks and takes reasonable paths around the obstacles but it can not learn an optimal path to collect the objects quickly, and it struggles with visually diverse rooms. As shown in video 3 below, the scripted policy gets distracted by the white patterned carpet while trying to locate more white objects to grasp.

1)
2)
3)
4)
We show a comparison between (1) our policy at the beginning of training (2) our policy at the end of training (3) the scripted policy. In (4) we can see the robot’s performance improve over time, and eventually exceed the scripted policy at quickly collecting the objects in the room.

Given we can use experts to code this hand-engineered controller, what is the purpose of learning? An important limitation of hand-engineered controllers is that they are tuned for a particular task, for example, grasping white objects. When diverse objects are introduced, which differ in color and shape, the original tuning may no longer be optimal. Rather than requiring further hand-engineering, our learning-based method is able to adapt itself to various tasks by collecting its own experience.

However, the most important lesson is that even if the hand-engineered controller is capable, the learning agent eventually surpasses it given enough time. This learning process is itself autonomous and takes place while the robot is performing its job, making it comparatively inexpensive. This shows the capability of learning agents, which can also be thought of as working out a general way to perform an “expert manual tuning” process for any kind of task. Learning systems have the ability to create the entire control algorithm for the robot, and are not limited to tuning a few parameters in a script. The key step in this work allows these real-world learning systems to autonomously collect the data needed to enable the success of learning methods.

This post is based on the paper “Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulation”, presented at CoRL 2021. You can find more details in our paper, on our website and the on the video. We provide code to reproduce our experiments. We thank Sergey Levine for his valuable feedback on this blog post.

View Details

For the second part of our RoboHouse Interview Trilogy: The Working Life of the Robotics Engineer we speak with Wendel Postma, chief engineer at Project MARCH VIII. How does he resolve the conundrum of integration: getting a bunch of single-minded engineers to ultimately serve the needs of one single exoskeleton user? Rens van Poppel inquires.

Wendel oversees technical engineering quality, and shares responsible for on-time delivery within budget with the other project managers. He spends his days wandering around the Dream Hall on TU Delft Campus, encouraging his team to explore new avenues for developing the exoskeleton. What is possible within the time that we have? Can conflicting design solutions work together?

Bringing bad news is part of the chief engineer’s job.

There is no shortage of hobbies and activities for Chief Engineer, Wendel. Sitting still is something he can’t do, which is why outside of Project MARCH, he is doing a lot of sports. This year, Wendel is making sure the team has 1 exoskeleton at the end of the year instead of many different parts. He also communicates well within the team so all the technological advances are understood and with a class of yoga so everyone can relax again. Wendel has many different goals. For example, he later wants to work in the health industry and complete an Ironman. Source: Project MARCH website.

In daily life, Arnhem-based Project MARCH pilot Koen van Zeeland is an executive in laying fibreglass in the Utrecht area. He was diagnosed with a spinal cord injury in 2013. Koen is a hard worker and his phone is always ringing. Yet he likes to make time to have a drink with his friends in the pub. Besides the pub, you might also find him on the moors, where he likes to walk his dog Turbo. Koen is also super sporty. Besides working out three times a week, Koen is also an avid cyclist with the goal of cycling up the mountains in Austria on his handbike. Source: Project MARCH website.

Koen van Zeeland is the primary test user of the exoskeleton and has control over the movements he makes. Project MARCH therefore calls him the ‘pilot’ of the exoskeleton. As the twenty-seventh and perhaps most important team member, Koen is valued highly within Project MARCH VIII. Source: Project MARCH website.

Project MARCH is iterative enterprise.Most of its workplace drama comes from the urgency to deliver at least one significant improvement on the existing prototype. This year’s obsessions is weight; a lighter exoskeleton would require less power from both pilot and motors. Self-balancing would become easier to realise.

In order not to weaken the frame of the exoskeleton, there was a lot of enthusiasm to experiment with carbon fibre, which is both a light and strong material. Something, however, got in the way: the team struggled to find a pilot.

My job is making sure that in the end we don’t have 600 separate parts, but one exoskeleton.

“Having a test pilot is crucial if we are to reach our goals,” Wendel says. “Our current exoskeleton is built to fit the particular body shape of the person controlling it. The design is not yet adjustable to a different body shape. So it is crucial to get the pilot involved as quickly as possible.”

Not having a pilot was stressful for the entire team.Their dream of creating a self-balancing exoskeleton was in danger. Wendel had to step up: “As chief engineer you have to make tough decisions. Carbon fibre is strong, but not flexible and difficult to machine. That is why we switched to aluminium, because it is easier to modify even after it is finished.”

“It was a huge disappointment,” Wendel says. “Some of us had already finished trainings for carbon manufacturing. Carbon parts were already ordered. The team felt let down. We had spent a so much time on something that was now impossible – because of the delays caused by having no pilot.”

“I learnt that bringing bad news is part of the chief engineer’s job. The next step is to look at how to convert the engineers’ enthusiasm for carbon fibre into new solutions and to redeploy their personal qualities.”

Wendel says the job also taught him to consider a hundred things at the same time. And to make sacrifices. Project MARCH involves long workdays and maybe not seeing your friends and roommates as much as you would like.

As a naturally curious person, Wendel found out that curiosity must be complemented by grit to make it in robotics. You often need to go deeper and study in more detail to make a good decision. “It is hard work. However, that is also what makes the job so much fun. You work in such a highly motivated team.”

That is also what makes the job so much fun.

The carbon story ended well, though.When the team did found a pilot, hard-working Koen van Zeeland, the choice for aluminium as a base material paid off. Through a process of weight analysis, parts can now be optimised for an ever lighter exoskeleton.

The Project MARCH team continues to grow through setbacks and has doubled-down on their efforts to create the world’s first self-balancing exoskeleton. If they succeed, it will be a huge success for this unique way of running a business.

The post RoboHouse Interview Trilogy, Part II: Wendel Postma and Project MARCH appeared first on RoboHouse.

View Details

Conventional sheet metal manufacturing is highly inefficient for the low-volume production seen in the space industry. At Machina Labs, they developed a novel method of forming sheet metal using two robotic arms to bend the metal into different geometries. This method cuts down the time to produce large sheet metal parts from several months down to a few hours. Ed Mehr, Co-Founder and CEO of Machina Labs, explains this revolutionary manufacturing process.

Ed MehrEd Mehr is the co-founder and CEO of Machina Labs. He has an engineering background in smart manufacturing and artificial intelligence. In his previous position at Relativity Space, he led a team in charge of developing the world’s largest metal 3D printer. Relativity Space uses 3D printing to make rocket parts rapidly, and with the flexibility for multiple iterations. Ed previously was the CTO at Cloudwear (Now Averon), and has also worked at SpaceX, Google, and Microsoft.

Links* Machina Labs * Download mp3 * Subscribe to Robohub using iTunes, RSS, or Spotify * Support us on Patreon

View Details

Claire chatted to Professor Yang Gao from the University of Surrey all about space robotics and planetary exploration.

Yang Gao is Professor of Space Autonomous Systems and Founding Head of the STAR LAB that specializes in robotic sensing, perception, visual guidance, navigation, and control (GNC) and biomimetic mechanisms for industrial applications in extreme environments. She brings over 20 years of research experience in developing robotics and autonomous systems, in which she has been the principal investigator of over 30 inter/nationally teamed projects and involved in real-world mission development.

View Details

Top 5 Robot Trends 2023 © International Federation of Robotics

The stock of operational robots around the globe hit a new record of about 3.5 million units – the value of installations reached an estimated 15.7 billion USD. The International Federation of Robotics analyzes the top 5 trends shaping robotics and automation in 2023.

“Robots play a fundamental role in securing the changing demands of manufacturers around the world,” says Marina Bill, President of the International Federation of Robotics. “New trends in robotics attract users from small enterprise to global OEMs.”

1 – Energy EfficiencyEnergy efficiency is key to improve companies’ competitiveness amid rising energy costs. The adoption of robotics helps in many ways to lower energy consumption in manufacturing. Compared to traditional assembly lines, considerable energy savings can be achieved through reduced heating. At the same time, robots work at high speed thus increasing production rates so that manufacturing becomes more time- and energy-efficient.

Today’s robots are designed to consume less energy, which leads to lower operating costs. To meet sustainability targets for their production, companies use industrial robots equipped with energy saving technology: robot controls are able to convert kinetic energy into electricity, for example, and feed it back into the power grid. This technology significantly reduces the energy required to run a robot. Another feature is the smart power saving mode that controls the robot´s energy supply on-demand throughout the workday. Since industrial facilities need to monitor their energy consumption even today, such connected power sensors are likely to become an industry standard for robotic solutions.

2 – ReshoringResilience has become an important driver for reshoring in various industries: Car manufacturers e.g. invest heavily in short supply lines to bring processes closer to their customers. These manufacturers use robot automation to manufacture powerful batteries cost-effectively and in large quantities to support their electric vehicle projects. These investments make the shipment of heavy batteries redundant. This is important as more and more logistics companies refuse to ship batteries for safety reasons.

Relocating microchip production back to the US and Europe is another reshoring trend. Since most industrial products nowadays require a semiconductor chip to function, their supply close to the customer is crucial. Robots play a vital role in chip manufacturing, as they live up to the extreme requirements of precision. Specifically designed robots automate the silicon wafer fabrication, take over cleaning and cleansing tasks or test integrated circuits. Recent examples of reshoring are Intel´s new chip factories in Ohio or the recently announced chip plant in the Saarland region of Germany run by chipmaker Wolfspeed and automotive supplier ZF.

3 – Robots easier to useRobot programming has become easier and more accessible to non-experts. Providers of software-driven automation platforms support companies, letting users manage industrial robots with no prior programming experience. Original equipment manufacturers work hand-in-hand with low code or even no-code technology partners that allow users of all skill levels to program a robot.

The easy-to-use software paired with an intuitive user experience replaces extensive robotics programming and opens up new robotics automation opportunities: Software start-ups are entering this market with specialized solutions for the needs of small and medium-sized companies. For example: a traditional heavy-weight industrial robot can be equipped with sensors and a new software that allows collaborative setup operation. This makes it easy for workers to adjust heavy machinery to different tasks. Companies will thus get the best of both worlds: robust and precise industrial robot hardware and state-of-the-art cobot software.

Easy-to-use programming interfaces, that allow customers to set up the robots themselves, also drive the emerging new segment of low-cost robotics. Many new customers reacted to the pandemic in 2020 by trying out robotic solutions. Robot suppliers acknowledged this demand: Easy setup and installation, for instance, with pre-configured software to handle grippers, sensors or controllers support lower-cost robot deployment. Such robots are often sold through web shops and program routines for various applications are downloadable from an app store.

4 – Artificial Intelligence (AI) and digital automationPropelled by advances in digital technologies, robot suppliers and system integrators offer new applications and improve existing ones regarding speed and quality. Connected robots are transforming manufacturing. Robots will increasingly operate as part of a connected digital ecosystem: Cloud Computing, Big Data Analytics or 5G mobile networks provide the technological base for optimized performance. The 5G standard will enable fully digitalized production, making cables on the shopfloor obsolete.

Artificial Intelligence (AI) holds great potential for robotics, enabling a range of benefits in manufacturing. The main aim of using AI in robotics is to better manage variability and unpredictability in the external environment, either in real-time, or off-line. This makes AI supporting machine learning play an increasing role in software offerings where running systems benefit, for example with optimized processes, predictive maintenance or vision-based gripping.

This technology helps manufacturers, logistics providers and retailers dealing with frequently changing products, orders and stock. The greater the variability and unpredictability of the environment, the more likely it is that AI algorithms will provide a cost-effective and fast solution – for example, for manufacturers or wholesalers dealing with millions of different products that change on a regular basis. AI is also useful in environments in which mobile robots need to distinguish between the objects or people they encounter and respond differently.

5 – Second life for industrial robotsSince an industrial robot has a service lifetime of up to thirty years, new tech equipment is a great opportunity to give old robots a “second life”. Industrial robot manufacturers like ABB, Fanuc, KUKA or Yaskawa run specialized repair centers close to their customers to refurbish or upgrade used units in a resource-efficient way. This prepare-to-repair strategy for robot manufacturers and their customers also saves costs and resources. To offer long-term repair to customers is an important contribution to the circular economy.

View Details

“I observed assistive technologies — developed by scientists and engineers my friends and I never met — which liberated us. My dream has always been to be one of those engineers.” Hermus says. Credit: Tony Pulsone

By Michaela Jarvis | Department of Mechanical Engineering

Before James Hermus started elementary school, he was a happy, curious kid who loved to learn. By the end of first grade, however, all that started to change, he says. As his schoolbooks became more advanced, Hermus could no longer memorize the words on each page, and pretend to be reading. He clearly knew the material the teacher presented in class; his teachers could not understand why he was unable to read and write his assignments. He was accused of being lazy and not trying hard enough.

Hermus was fortunate to have parents who sought out neuropsychology testing — which documented an enormous discrepancy between his native intelligence and his symbol decoding and phonemic awareness. Yet despite receiving a diagnosis of dyslexia, Hermus and his family encountered resistance at his school. According to Hermus, the school’s reading specialist did not “believe” in dyslexia, and, he says, the principal threatened his family with truancy charges when they took him out of school each day to attend tutoring.

Hermus’ school, like many across the country, was reluctant to provide accommodations for students with learning disabilities who were not two years behind in two subjects, Hermus says. For this reason, obtaining and maintaining accommodations, such as extended time and a reader, was a constant battle from first through 12th grade: Students who performed well lost their right to accommodations. Only through persistence and parental support did Hermus succeed in an educational system which he says all too often fails students with learning disabilities.

By the time Hermus was in high school, he had to become a strong self-advocate. In order to access advanced courses, he needed to be able to read more and faster, so he sought out adaptive technology — Kurzweil, a text-to-audio program. This, he says, was truly life-changing. At first, to use this program he had to disassemble textbooks, feed the pages through a scanner, and digitize them.

After working his way to the University of Wisconsin at Madison, Hermus found a research opportunity in medical physics and then later in biomechanics. Interestingly, the steep challenges that Hermus faced during his education had developed in him “the exact skill set that makes a successful researcher,” he says. “I had to be organized, advocate for myself, seek out help to solve problems that others had not seen before, and be excessively persistent.”

While working as a member of Professor Darryl Thelen’s Neuromuscular Biomechanics Lab at Madison, Hermus helped design and test a sensor for measuring tendon stress. He recognized his strengths in mechanical design. During this undergraduate research, he co-authored numerous journal and conference papers. These experiences and a desire to help people with physical disabilities propelled him to MIT.

“MIT is an incredible place. The people in MechE at MIT are extremely passionate and unassuming. I am not unusual at MIT,” Hermus says. Credit: Tony Pulsone

In September 2022, Hermus completed his PhD in mechanical engineering from MIT. He has been an author on seven papers in peer-reviewed journals, three as first author and four of them published when he was an undergraduate. He has won awards for his academics and for his mechanical engineering research and has served as a mentor and an advocate for disability awareness in several different contexts.

His work as a researcher stems directly from his personal experience, Hermus says. As a student in a special education classroom, “I observed assistive technologies — developed by scientists and engineers my friends and I never met — which liberated us. My dream has always been to be one of those engineers.”

Hermus’ work aims to investigate and model human interaction with objects where both substantial motion and force are present. His research has demonstrated that the way humans perform such everyday actions as turning a steering wheel or opening a door is very different from much of robotics. He showed specific patterns exist in the behavior that provide insight into neural control. In 2020, Hermus was the first author on a paper on this topic, which was published in the Journal of Neurophysiology and later won first place in the MIT Mechanical Engineering Research Exhibition. Using this insight, Hermus and his colleagues implemented these strategies on a Kuka LBR iiwa robot to learn about how humans regulate their many degrees of freedom. This work was published in IEEE Transactions on Robotics 2022. More recently, Hermus has collaborated with researchers at the University of Pittsburgh to see if these ideas prove useful in the development of brain computer interfaces — using electrodes implanted in the brain to control a prosthetic robotic arm.

While the hardware of prosthetics and exoskeletons is advancing, Hermus says, there are daunting limitations to the field in the descriptive modeling of human physical behavior, especially during contact with objects. Without these descriptive models, developing generalizable implementations of prosthetics, exoskeletons, and rehabilitation robotics will prove challenging.

“We need competent descriptive models of human physical interaction,” he says.

While earning his master’s and doctoral degrees at MIT, Hermus worked with Neville Hogan, the Sun Jae Professor of Mechanical Engineering, in the Eric P. and Evelyn E. Newman Laboratory for Biomechanics and Human Rehabilitation. Hogan has high praise for the research Hermus has conducted over his six years in the Newman lab.

“James has done superb work for both his master’s and doctoral theses. He tackled a challenging problem and made excellent and timely progress towards its solution. He was a key member of my research group,” Hogan says. “James’ commitment to his research is unquestionably a reflection of his own experience.”

Following postdoctoral research at MIT, where he has also been a part-time lecturer, Hermus is now beginning postdoctoral work with Professor Aude Billard at EPFL in Switzerland, where he hopes to gain experience with learning and optimization methods to further his human motor control research.

Hermus’ enthusiasm for his research is palpable, and his zest for learning and life shines through despite the hurdles his dyslexia presented. He demonstrates a similar kind of excitement for ski-touring and rock-climbing with the MIT Outing Club, working at MakerWorkshop, and being a member of the MechE community.

“MIT is an incredible place. The people in MechE at MIT are extremely passionate and unassuming. I am not unusual at MIT,” he says. “Nearly every person I know well has a unique story with an unconventional path.”

View Details

Part one of our RoboHouse Interview Trilogy: The Working Life of Robotics Engineers seeks out Christian Geckeler. Christian is a PhD student at the Environmental Robotics Lab of ETH Zürich. He speaks with Rens van Poppel about the experience of getting high into the wild.

What if drones could help place sensors in forests more easily? What if a sensor device could automatically grab and hold a tree branch? Which flexible material is also strong and biodegradable? These leaps of imagination lead Christian to a new kind of gripper, inspired by the Japanese art of folding.

His origami design wraps itself around tree branches close enough to trigger an unfolding movement. This invention may in the future improve our insight into hard-to-access forest canopies, in a way that is environmentally friendly and pleasant for human operators.

What is it like to work in the forest as a researcher with this technology?“Robotic solutions deployed in forests are currently scarce,” says Christian. “So developing solutions for such an environment is challenging, but also rewarding. Personally I also enjoy being outdoors. Compared to a lab, the forest is wilder and more unpredictable. Which I find wonderful, except when it’s cold.”

Are there limits as to where the gripper can be deployed?
“The gripper is quite versatile. Rather than the type of trees, it is the diameter and angle of the branch that dictate whether the gripper can attach. Even so, dense foliage could hinder the drone, and there should be sufficient space for the gripper to attach.”

Christian Geckeler, PhD student at the Environmental Robotics Lab of ETH Zürich, a university for science and technology in Switzerland where some 530 professors teach around 20,500 students – including 4,100 doctoral students – from over 120 countries.Are the used materials environmentally friendly?“Currently not all components are biodegradable, and the gripper must be recollected after sampling is finished. However, we are currently working on a fully biodegradable gripper, which releases itself and falls on the ground after being exposed to sufficient amounts of water, which makes collection much easier.”

How good at outdoor living do aspiring tree-canopy researchers need to be?
“Everything is a learning process,” says Christian philosophically. “Rather than existing expertise, a willingness to learn and passion for the subject is much more important.”

What happens when the drone gets stuck in a tree?
“As a safety measure, the drone has a protective net on top which prevents leaves and branches from coming in contact with the propeller. And we avoid interaction between the drone and foliage, so this has never happened.”

What struck you when took the gripper into the wild?
“Perhaps the most surprising thing was the great variance that is found in nature; no two trees are alike and every branch is different. The only way of finding out if your solution works is by testing outside as soon and as often as possible.”

Christian ends with a note on the importance of social and technical interplay in robotics: “You may think you develop a robot perfectly, but you must make sure society actually wants it and that it is easy to use for not technically-minded people too.”

The post RoboHouse Interview Trilogy, Part I: Christian Geckeler and The Origami Gripper appeared first on RoboHouse.

View Details

Claire chatted to Professor Ignazio Maria Viola from the University of Edinburgh all about aerodynamics, dandelion-inspired drones, and swarm sensing.

Ignazio Maria Viola is Professor of Fluid Mechanics and Bioinspired Engineering at the School of Engineering, University of Edinburgh, and Fellow of the Royal Institution of Naval Architects. He is the recipient of the ERC Consolidator Grant Dandidrone to explore the unsteady aerodynamics of dandelion-inspired drones.

View Details

Researchers have come up with an innovative approach to building deformable underwater robots using simple repeating substructures. The team has demonstrated the new system in two different example configurations, one like an eel, pictured here in the MIT tow tank. Credit: Courtesy of the researchers

By David L. Chandler | MIT News Office

Underwater structures that can change their shapes dynamically, the way fish do, push through water much more efficiently than conventional rigid hulls. But constructing deformable devices that can change the curve of their body shapes while maintaining a smooth profile is a long and difficult process. MIT’s RoboTuna, for example, was composed of about 3,000 different parts and took about two years to design and build.

Now, researchers at MIT and their colleagues — including one from the original RoboTuna team — have come up with an innovative approach to building deformable underwater robots, using simple repeating substructures instead of unique components. The team has demonstrated the new system in two different example configurations, one like an eel and the other a wing-like hydrofoil. The principle itself, however, allows for virtually unlimited variations in form and scale, the researchers say.

The work is being reported in the journal Soft Robotics, in a paper by MIT research assistant Alfonso Parra Rubio, professors Michael Triantafyllou and Neil Gershenfeld, and six others.

Existing approaches to soft robotics for marine applications are generally made on small scales, while many useful real-world applications require devices on scales of meters. The new modular system the researchers propose could easily be extended to such sizes and beyond, without requiring the kind of retooling and redesign that would be needed to scale up current systems.

The deformable robots are made with lattice-like pieces, called voxels, that are low density and have high stiffness. The deformable robots are made with lattice-like pieces, called voxels, that are low density and have high stiffness. Credit: Courtesy of the researchers

“Scalability is a strong point for us,” says Parra Rubio. Given the low density and high stiffness of the lattice-like pieces, called voxels, that make up their system, he says, “we have more room to keep scaling up,” whereas most currently used technologies “rely on high-density materials facing drastic problems” in moving to larger sizes.

The individual voxels in the team’s experimental, proof-of-concept devices are mostly hollow structures made up of cast plastic pieces with narrow struts in complex shapes. The box-like shapes are load-bearing in one direction but soft in others, an unusual combination achieved by blending stiff and flexible components in different proportions.

“Treating soft versus hard robotics is a false dichotomy,” Parra Rubio says. “This is something in between, a new way to construct things.” Gershenfeld, head of MIT’s Center for Bits and Atoms, adds that “this is a third way that marries the best elements of both.”

“Smooth flexibility of the body surface allows us to implement flow control that can reduce drag and improve propulsive efficiency, resulting in substantial fuel saving,” says Triantafyllou, who is the Henry L. and Grace Doherty Professor in Ocean Science and Engineering, and was part of the RoboTuna team.

Credit: Courtesy of the researchers.

In one of the devices produced by the team, the voxels are attached end-to-end in a long row to form a meter-long, snake-like structure. The body is made up of four segments, each consisting of five voxels, with an actuator in the center that can pull a wire attached to each of the two voxels on either side, contracting them and causing the structure to bend. The whole structure of 20 units is then covered with a rib-like supporting structure, and then a tight-fitting waterproof neoprene skin. The researchers deployed the structure in an MIT tow tank to show its efficiency in the water, and demonstrated that it was indeed capable of generating forward thrust sufficient to propel itself forward using undulating motions.

“There have been many snake-like robots before,” Gershenfeld says. “But they’re generally made of bespoke components, as opposed to these simple building blocks that are scalable.”

For example, Parra Rubio says, a snake-like robot built by NASA was made up of thousands of unique pieces, whereas for this group’s snake, “we show that there are some 60 pieces.” And compared to the two years spent designing and building the MIT RoboTuna, this device was assembled in about two days, he says.

The individual voxels are mostly hollow structures made up of cast plastic pieces with narrow struts in complex shapes. Credit: Courtesy of the researchers

The other device they demonstrated is a wing-like shape, or hydrofoil, made up of an array of the same voxels but able to change its profile shape and therefore control the lift-to-drag ratio and other properties of the wing. Such wing-like shapes could be used for a variety of purposes, ranging from generating power from waves to helping to improve the efficiency of ship hulls — a pressing demand, as shipping is a significant source of carbon emissions.

The wing shape, unlike the snake, is covered in an array of scale-like overlapping tiles, designed to press down on each other to maintain a waterproof seal even as the wing changes its curvature. One possible application might be in some kind of addition to a ship’s hull profile that could reduce the formation of drag-inducing eddies and thus improve its overall efficiency, a possibility that the team is exploring with collaborators in the shipping industry.

The team also created a wing-like hydrofoil. Credit: Courtesy of the researchers

Ultimately, the concept might be applied to a whale-like submersible craft, using its morphable body shape to create propulsion. Such a craft that could evade bad weather by staying below the surface, but without the noise and turbulence of conventional propulsion. The concept could also be applied to parts of other vessels, such as racing yachts, where having a keel or a rudder that could curve gently during a turn instead of remaining straight could provide an extra edge. “Instead of being rigid or just having a flap, if you can actually curve the way fish do, you can morph your way around the turn much more efficiently,” Gershenfeld says.

The research team included Dixia Fan of the Westlake University in China; Benjamin Jenett SM ’15, PhD ’ 20 of Discrete Lattice Industries; Jose del Aguila Ferrandis, Amira Abdel-Rahman and David Preiss of MIT; and Filippos Tourlomousis of the Demokritos Research Center of Greece. The work was supported by the U.S. Army Research Lab, CBA Consortia funding, and the MIT Sea Grant Program.

• PAPER – Modular Morphing Lattices for Large-Scale Underwater Continuum Robotic Structures. Alfonso Parra Rubio, Dixia Fan, Benjamin Jenett, José del Águila Ferrandis, Filippos Tourlomousis, Amira Abdel-Rahman, David Preiss, Michael Triantafyllou, and Neil Gershenfeld. Soft Robotics (2023).

View Details

A photograph of an eBiobot prototype, lit with blue microLEDs. Remotely controlled miniature biological robots have many potential applications in medicine, sensing and environmental monitoring. Image courtesy of Yongdeok Kim

By Liz Ahlberg Touchstone

First, they walked. Then, they saw the light. Now, miniature biological robots have gained a new trick: remote control.

The hybrid “eBiobots” are the first to combine soft materials, living muscle and microelectronics, said researchers at the University of Illinois Urbana-Champaign, Northwestern University and collaborating institutions. They described their centimeter-scale biological machines in the journal Science Robotics.

“Integrating microelectronics allows the merger of the biological world and the electronics world, both with many advantages of their own, to now produce these electronic biobots and machines that could be useful for many medical, sensing and environmental applications in the future,” said study co-leader Rashid Bashir, an Illinois professor of bioengineering and dean of the Grainger College of Engineering.

Rashid Bashir. Photo by L. Brian Stauffer

Bashir’s group has pioneered the development of biobots, small biological robots powered by mouse muscle tissue grown on a soft 3D-printed polymer skeleton. They demonstrated walking biobots in 2012 and light-activated biobots in 2016. The light activation gave the researchers some control, but practical applications were limited by the question of how to deliver the light pulses to the biobots outside of a lab setting.

The answer to that question came from Northwestern University professor John A. Rogers, a pioneer in flexible bioelectronics, whose team helped integrate tiny wireless microelectronics and battery-free micro-LEDs. This allowed the researchers to remotely control the eBiobots.

“This unusual combination of technology and biology opens up vast opportunities in creating self-healing, learning, evolving, communicating and self-organizing engineered systems. We feel that it’s a very fertile ground for future research with specific potential applications in biomedicine and environmental monitoring,” said Rogers, a professor of materials science and engineering, biomedical engineering and neurological surgery at Northwestern University and director of the Querrey Simpson Institute for Bioelectronics.

Remote control steering allows the eBiobots to maneuver around obstacles, as shown in this composite image of a bipedal robot traversing a maze. Image courtesy of Yongdeok Kim

To give the biobots the freedom of movement required for practical applications, the researchers set out to eliminate bulky batteries and tethering wires. The eBiobots use a receiver coil to harvest power and provide a regulated output voltage to power the micro-LEDs, said co-first author Zhengwei Li, an assistant professor of biomedical engineering at the University of Houston.

The researchers can send a wireless signal to the eBiobots that prompts the LEDs to pulse. The LEDs stimulate the light-sensitive engineered muscle to contract, moving the polymer legs so that the machines “walk.” The micro-LEDs are so targeted that they can activate specific portions of muscle, making the eBiobot turn in a desired direction. See a video on YouTube.

The researchers used computational modeling to optimize the eBiobot design and component integration for robustness, speed and maneuverability. Illinois professor of mechanical sciences and engineering Mattia Gazzola led the simulation and design of the eBiobots. The iterative design and additive 3D printing of the scaffolds allowed for rapid cycles of experiments and performance improvement, said Gazzola and co-first author Xiaotian Zhang, a postdoctoral researcher in Gazzola’s lab.

The eBiobots are the first wireless bio-hybrid machines, combining biological tissue, microelectronics and 3D-printed soft polymers. Image courtesy of Yongdeok Kim

The design allows for possible future integration of additional microelectronics, such as chemical and biological sensors, or 3D-printed scaffold parts for functions like pushing or transporting things that the biobots encounter, said co-first author Youngdeok Kim, who completed the work as a graduate student at Illinois.

The integration of electronic sensors or biological neurons would allow the eBiobots to sense and respond to toxins in the environment, biomarkers for disease and more possibilities, the researchers said.

“In developing a first-ever hybrid bioelectronic robot, we are opening the door for a new paradigm of applications for health care innovation, such as in-situ biopsies and analysis, minimum invasive surgery or even cancer detection within the human body,” Li said.

The National Science Foundation and the National Institutes of Health supported this work.

• PAPER – Remote control of muscle-driven miniature robots with battery-free wireless optoelectronics. Yongdeok Kim, Yiyuan Yang, Xiaotian Zhang,Zhengwei Li, Abraham Vázquez-Guardado, Insu Park, Jiaojiao Wang, Andrew I. Efimov, Zhi Dou, Yue Wang, Junehu Park, Haiwen Luan, Xinchen Ni, Yun Seong Kim, Janice Baek, Joshua Jaehyung Park, Zhaoqian Xie, Hangbo Zhao, Mattia Gazzola, John A. Rogers, and Rashid Bashir. Science Robotics 8.74 (2023): eadd1053.

View Details

Claire chatted to Professor Emily S. Cross from the University of Glasgow and Western Sydney University all about neuroscience, social learning, and human-robot interaction.

Emily S. Cross is a Professor of Social Robotics at the University of Glasgow, and a Professor of Human Neuroscience at the MARCS Institute at Western Sydney University. Using interactive learning tasks, brain scanning, and dance, acrobatics and robots, she and her Social Brain in Action Laboratory team explore how we learn by watching others throughout the lifespan, how action experts’ brains enable them to perform physical skills so exquisitely, and the social influences that shape human-robot interaction.

View Details

RoboSalps in action. Credits: Valentina Lo Gatto

These robotic units called RoboSalps, after their animal namesakes, have been engineered to operate in unknown and extreme environments such as extra-terrestrial oceans.

Although salps resemble jellyfish with their semi-transparent barrel-shaped bodies, they belong to the family of Tunicata and have a complex life cycle, changing between solitary and aggregate generations where they connect to form colonies.

RoboSalps have similarly light, tubular bodies and can link to each other to form ‘colonies’ which gives them new capabilities that can only be achieved because they work together.

Researcher Valentina Lo Gatto of Bristol’s Department of Aerospace Engineering is leading the study. She is also a student at the EPSRC Centre of Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE CDT).

She said: “RoboSalp is the first modular salp-inspired robot. Each module is made of a very light-weight soft tubular structure and a drone propeller which enables them to swim. These simple modules can be combined into ‘colonies’ that are much more robust and have the potential to carry out complex tasks. Because of their low weight and their robustness, they are ideal for extra-terrestrial underwater exploration missions, for example, in the subsurface ocean on the Jupiter moon Europa.”

RoboSalps are unique as each individual module can swim on its own. This is possible because of a small motor with rotor blades – typically used for drones – inserted into the soft tubular structure.

When swimming on their own, RoboSalps modules are difficult to control, but after joining them together to form colonies, they become more stable and show sophisticated movements.

In addition, by having multiple units joined together, scientists automatically obtain a redundant system, which makes it more robust against failure. If one module breaks, the whole colony can still move.

A colony of soft robots is a relatively novel concept with a wide range of interesting applications. RoboSalps are soft, potentially quite energy efficient, and robust due to inherent redundancy. This makes them ideal for autonomous missions where a direct and immediate human control might not be feasible.

Dr Helmut Hauser of Bristol’s Department of Engineering Maths, explained: “These include the exploration of remote submarine environments, sewage tunnels, and industrial cooling systems. Due to the low weight and softness of the RoboSalp modules, they are also ideal for extra-terrestrial missions. They can easily be stored in a reduced volume, ideal for reducing global space mission payloads.”

A compliant body also provides safer interaction with potentially delicate ecosystems, both on earth and extra-terrestrial, reducing the risk of environmental damage. The possibility to detach units or segments, and rearrange them, gives the system adaptability: once the target environment is reached, the colony could be deployed to start its exploration.

At a certain point, it could split into multiple segments, each exploring in a different direction, and afterwards reassemble in a new configuration to achieve a different objective such as manipulation or sample collection.

Prof Jonathan Rossiter added: “We are also developing control approaches that are able to exploit the compliance of the modules with the goal of achieving energy efficient movements close to those observed in biological salps.”

View Details

Robotic head, 3D illustration (frank60/Shutterstock)

By Jonathan Roberts (Professor in Robotics, Queensland University of Technology)

With generative artificial intelligence (AI) systems such as ChatGPT and StableDiffusion being the talk of the town right now, it might feel like we’ve taken a giant leap closer to a sci-fi reality where AIs are physical entities all around us.

Indeed, computer-based AI appears to be advancing at an unprecedented rate. But the rate of advancement in robotics – which we could think of as the potential physical embodiment of AI – is slow.

Could it be that future AI systems will need robotic “bodies” to interact with the world? If so, will nightmarish ideas like the self-repairing, shape-shifting T-1000 robot from the Terminator 2 movie come to fruition? And could a robot be created that could “live” forever?

Energy for ‘life’Biological lifeforms like ourselves need energy to operate. We get ours via a combination of food, water, and oxygen. The majority of plants also need access to light to grow.

By the same token, an everlasting robot needs an ongoing energy supply. Currently, electrical power dominates energy supply in the world of robotics. Most robots are powered by the chemistry of batteries.

An alternative battery type has been proposed that uses nuclear waste and ultra-thin diamonds at its core. The inventors, a San Francisco startup called Nano Diamond Battery, claim a possible battery life of tens of thousands of years. Very small robots would be an ideal user of such batteries.

But a more likely long-term solution for powering robots may involve different chemistry – and even biology. In 2021, scientists from the Berkeley Lab and UMAss Amherst in the US demonstrated tiny nanobots could get their energy from chemicals in the liquid they swim in.

The researchers are now working out how to scale up this idea to larger robots that can work on solid surfaces.

Repairing and copying oneselfOf course, an undying robot might still need occasional repairs.

Ideally, a robot would repair itself if possible. In 2019, a Japanese research group demonstrated a research robot called PR2 tightening its own screw using a screwdriver. This is like self-surgery! However, such a technique would only work if non-critical components needed repair.

Other research groups are exploring how soft robots can self-heal when damaged. A group in Belgium showed how a robot they developed recovered after being stabbed six times in one of its legs. It stopped for a few minutes until its skin healed itself, and then walked off.

Another unusual concept for repair is to use other things a robot might find in the environment to replace its broken part.

Last year, scientists reported how dead spiders can be used as robot grippers. This form of robotics is known as “necrobotics”. The idea is to use dead animals as ready-made mechanical devices and attach them to robots to become part of the robot.

The proof-of-concept in necrobotics involved taking a dead spider and ‘reanimating’ its hydraulic legs with air, creating a surprisingly strong gripper. Preston Innovation Laboratory/Rice University

A robot colony?From all these recent developments, it’s quite clear that in principle, a single robot may be able to live forever. But there is a very long way to go.

Most of the proposed solutions to the energy, repair and replication problems have only been demonstrated in the lab, in very controlled conditions and generally at tiny scales.

The ultimate solution may be one of large colonies or swarms of tiny robots who share a common brain, or mind. After all, this is exactly how many species of insects have evolved.

The concept of the “mind” of an ant colony has been pondered for decades. Research published in 2019 showed ant colonies themselves have a form of memory that is not contained within any of the ants.

This idea aligns very well with one day having massive clusters of robots that could use this trick to replace individual robots when needed, but keep the cluster “alive” indefinitely.

Ant colonies can contain ‘memories’ that are distributed between many individual insects. frank60/Shutterstock

Ultimately, the scary robot scenarios outlined in countless science fiction books and movies are unlikely to suddenly develop without anyone noticing.

Engineering ultra-reliable hardware is extremely difficult, especially with complex systems. There are currently no engineered products that can last forever, or even for hundreds of years. If we do ever invent an undying robot, we’ll also have the chance to build in some safeguards.

Jonathan Roberts is Director of the Australian Cobotics Centre, the Technical Director of the Advanced Robotics for Manufacturing (ARM) Hub, and is a Chief Investigator at the QUT Centre for Robotics. He receives funding from the Australian Research Council. He was the co-founder of the UAV Challenge – an international drone competition.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

View Details

Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and the Media Lab, seeks to develop wireless technology that can sense the physical world in ways that were not possible before. Image: Adam Glanzman

By Adam Zewe | MIT News Office

Fadel Adib never expected that science would get him into the White House, but in August 2015 the MIT graduate student found himself demonstrating his research to the president of the United States.

Adib, fellow grad student Zachary Kabelac, and their advisor, Dina Katabi, showcased a wireless device that uses Wi-Fi signals to track an individual’s movements.

As President Barack Obama looked on, Adib walked back and forth across the floor of the Oval Office, collapsed onto the carpet to demonstrate the device’s ability to monitor falls, and then sat still so Katabi could explain to the president how the device was measuring his breathing and heart rate.

“Zach started laughing because he could see that my heart rate was 110 as I was demoing the device to the president. I was stressed about it, but it was so exciting. I had poured a lot of blood, sweat, and tears into that project,” Adib recalls.

For Adib, the White House demo was an unexpected — and unforgettable — culmination of a research project he had launched four years earlier when he began his graduate training at MIT. Now, as a newly tenured associate professor in the Department of Electrical Engineering and Computer Science and the Media Lab, he keeps building off that work. Adib, the Doherty Chair of Ocean Utilization, seeks to develop wireless technology that can sense the physical world in ways that were not possible before.

In his Signal Kinetics group, Adib and his students apply knowledge and creativity to global problems like climate change and access to health care. They are using wireless devices for contactless physiological sensing, such as measuring someone’s stress level using Wi-Fi signals. The team is also developing battery-free underwater cameras that could explore uncharted regions of the oceans, tracking pollution and the effects of climate change. And they are combining computer vision and radio frequency identification (RFID) technology to build robots that find hidden items, to streamline factory and warehouse operations and, ultimately, alleviate supply chain bottlenecks.

While these areas may seem quite different, each time they launch a new project, the researchers uncover common threads that tie the disciplines together, Adib says.

“When we operate in a new field, we get to learn. Every time you are at a new boundary, in a sense you are also like a kid, trying to understand these different languages, bring them together, and invent something,” he says.

A science-minded childA love of learning has driven Adib since he was a young child growing up in Tripoli on the coast of Lebanon. He had been interested in math and science for as long as he could remember, and had boundless energy and insatiable curiosity as a child.

“When my mother wanted me to slow down, she would give me a puzzle to solve,” he recalls.

By the time Adib started college at the American University of Beirut, he knew he wanted to study computer engineering and had his sights set on MIT for graduate school.

Seeking to kick-start his future studies, Adib reached out to several MIT faculty members to ask about summer internships. He received a response from the first person he contacted. Katabi, the Thuan and Nicole Pham Professor in the Department of Electrical Engineering and Computer Science (EECS), and a principal investigator in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the MIT Jameel Clinic, interviewed him and accepted him for a position. He immersed himself in the lab work and, as the end of summer approached, Katabi encouraged him to apply for grad school at MIT and join her lab.

“To me, that was a shock because I felt this imposter syndrome. I thought I was moving like a turtle with my research, but I did not realize that with research itself, because you are at the boundary of human knowledge, you are expected to progress iteratively and slowly,” he says.

As an MIT grad student, he began contributing to a number of projects. But his passion for invention pushed him to embark into unexplored territory. Adib had an idea: Could he use Wi-Fi to see through walls?

“It was a crazy idea at the time, but my advisor let me work on it, even though it was not something the group had been working on at all before. We both thought it was an exciting idea,” he says.

As Wi-Fi signals travel in space, a small part of the signal passes through walls — the same way light passes through windows — and is then reflected by whatever is on the other side. Adib wanted to use these signals to “see” what people on the other side of a wall were doing.

Discovering new applicationsThere were a lot of ups and downs (“I’d say many more downs than ups at the beginning”), but Adib made progress. First, he and his teammates were able to detect people on the other side of a wall, then they could determine their exact location. Almost by accident, he discovered that the device could be used to monitor someone’s breathing.

“I remember we were nearing a deadline and my friend Zach and I were working on the device, using it to track people on the other side of the wall. I asked him to hold still, and then I started to see him appearing and disappearing over and over again. I thought, could this be his breathing?” Adib says.

Eventually, they enabled their Wi-Fi device to monitor heart rate and other vital signs. The technology was spun out into a startup, which presented Adib with a conundrum once he finished his PhD — whether to join the startup or pursue a career in academia.

He decided to become a professor because he wanted to dig deeper into the realm of invention. But after living through the winter of 2014-2015, when nearly 109 inches of snow fell on Boston (a record), Adib was ready for a change of scenery and a warmer climate. He applied to universities all over the United States, and while he had some tempting offers, Adib ultimately realized he didn’t want to leave MIT. He joined the MIT faculty as an assistant professor in 2016 and was named associate professor in 2020.

“When I first came here as an intern, even though I was thousands of miles from Lebanon, I felt at home. And the reason for that was the people. This geekiness — this embrace of intellect — that is something I find to be beautiful about MIT,” he says.

He’s thrilled to work with brilliant people who are also passionate about problem-solving. The members of his research group are diverse, and they each bring unique perspectives to the table, which Adib says is vital to encourage the intellectual back-and-forth that drives their work.

Diving into a new projectFor Adib, research is exploration. Take his work on oceans, for instance. He wanted to make an impact on climate change, and after exploring the problem, he and his students decided to build a battery-free underwater camera.

Adib learned that the ocean, which covers 70 percent of the planet, plays the single largest role in the Earth’s climate system. Yet more than 95 percent of it remains unexplored. That seemed like a problem the Signal Kinetics group could help solve, he says.

But diving into this research area was no easy task. Adib studies Wi-Fi systems, but Wi-Fi does not work underwater. And it is difficult to recharge a battery once it is deployed in the ocean, making it hard to build an autonomous underwater robot that can do large-scale sensing.

So, the team borrowed from other disciplines, building an underwater camera that uses acoustics to power its equipment and capture and transmit images.

“We had to use piezoelectric materials, which come from materials science, to develop transducers, which come from oceanography, and then on top of that we had to marry these things with technology from RF known as backscatter,” he says. “The biggest challenge becomes getting these things to gel together. How do you decode these languages across fields?”

It’s a challenge that continues to motivate Adib as he and his students tackle problems that are too big for one discipline.

He’s excited by the possibility of using his undersea wireless imaging technology to explore distant planets. These same tools could also enhance aquaculture, which could help eradicate food insecurity, or support other emerging industries.

To Adib, the possibilities seem endless.

“With each project, we discover something new, and that opens up a whole new world to explore. The biggest driver of our work in the future will be what we think is impossible, but that we could make possible,” he says.

View Details

Claire chatted to Dr Sabine Hauert from the University of Bristol all about swarm robotics, nanorobots, and environmental monitoring.

Sabine Hauert is Associate Professor of Swarm Engineering at University of Bristol. She leads a team of 20 researchers working on making swarms for people, and across scales, from nanorobots for cancer treatment, to larger robots for environmental monitoring, or logistics. Previously she worked at MIT and EPFL. She is President and Executive Trustee of non-profits robohub.org and aihub.org, which connect the robotics and AI communities to the public.

View Details

By Peter Rüegg

Ecologists are increasingly using traces of genetic material left behind by living organisms left behind in the environment, called environmental DNA (eDNA), to catalogue and monitor biodiversity. Based on these DNA traces, researchers can determine which species are present in a certain area.

Obtaining samples from water or soil is easy, but other habitats – such as the forest canopy – are difficult for researchers to access. As a result, many species remain untracked in poorly explored areas.

Researchers at ETH Zurich and the Swiss Federal Institute for Forest, Snow and Landscape Research WSL, and the company SPYGEN have partnered to develop a special drone that can autonomously collect samples on tree branches.

(Video: ETH Zürich)

How the drone collects materialThe drone is equipped with adhesive strips. When the aircraft lands on a branch, material from the branch sticks to these strips. Researchers can then extract DNA in the lab, analyse it and assign it to genetic matches of the various organisms using database comparisons.

But not all branches are the same: they vary in terms of their thickness and elasticity. Branches also bend and rebound when a drone lands on them. Programming the aircraft in such a way that it can still approach a branch autonomously and remain stable on it long enough to take samples was a major challenge for the roboticists.

“Landing on branches requires complex control,” explains Stefano Mintchev, Professor of Environmental Robotics at ETH Zurich and WSL. Initially, the drone does not know how flexible a branch is, so the researchers fitted it with a force sensing cage. This allows the drone to measure this factor at the scene and incorporate it into its flight manoeuvre.

Scheme: DNA is extracted from the collected branch material, amplified, sequenced and the sequences found are compared with databases. This allows the species to be identified. (Graphic: Stefano Mintchev / ETH Zürich)

Preparing rainforest operations at Zoo ZurichResearchers have tested their new device on seven tree species. In the samples, they found DNA from 21 distinct groups of organisms, or taxa, including birds, mammals and insects. “This is encouraging, because it shows that the collection technique works,“ says Mintchev, who co-​authored the study that has appeared in the journal Science Robotics.

The researchers now want to improve their drone further to get it ready for a competition in which the aim is to detect as many different species as possible across 100 hectares of rainforest in Singapore in 24 hours.

To test the drone’s efficiency under conditions similar to those it will experience at the competition, Mintchev and his team are currently working at the Zoo Zurich’s Masoala Rainforest. “Here we have the advantage of knowing which species are present, which will help us to better assess how thorough we are in capturing all eDNA traces with this technique or if we’re missing something,“ Mintchev says.

For this event, however, the collection device must become more efficient and mobilize faster. In the tests in Switzerland, the drone collected material from seven trees in three days; in Singapore, it must be able to fly to and collect samples from ten times as many trees in just one day.

Collecting samples in a natural rainforest, however, presents the researchers with even tougher challenges. Frequent rain washes eDNA off surfaces, while wind and clouds impede drone operation. “We are therefore very curious to see whether our sampling method will also prove itself under extreme conditions in the tropics,” Mintchev says.

• PAPER – Drone-​assisted collection of environmental DNA from tree branches for biodiversity monitoring. Aucone E, Kirchgeorg E, Valentini A, Pellissier L, Deiner K, and Mintchev S. Science Robotics, 18 January 2023. DOI 10.1126/scirobotics.add5762

View Details

Robots were on the main expo floor at CES this year, and these weren’t just cool robots for marketing purposes. I’ve been tracking robots at CES for more than 10 years, watching the transition from robot toys to real robots. Increasing prominence has been given to self-driving cars, LiDARs and eVTOL drones, but, in my mind it was really the inclusion of John Deere and agricultural robots last year that confirmed that CES was incorporating more industry, more real machines, not just gadgets.
In fact, according to the organizing association CTA or the Consumer Technology Association, these days CES no longer stands for the Consumer Electronics Show. CES now just stands for CES, one of the world’s largest technology expos.

Eve from Halodi Robotics shakes hands at CES 2023 with Karinne Ramirez-Amaro, associate professor at Chalmers University of Technology and head of IEEE Robotics and Automation Society’s Women in Engineering chapter. (Image source: Andra Keay)

The very first robot I saw was Eve from Halodi Robotics, exhibiting in the ADT Commercial booth. I am a big fan of this company. Not only do they have great robotics technology, which is very safe and collaborative, but I’ve watched them go from an angel funded startup to their first commercial deployments, providing 140 robots to ADT. One of their secrets has been spending the last year working closely with ADT to finetune the first production features of Eve, focusing on indoor security and working alongside people. In the future, Halodi has potential for many other applications including eldercare.

Another robot company (and robot) that I’m a big fan of is Labrador Robotics, and their mobile tray fetching robot for eldercare. Labrador exhibited their mobile robot in the AARP Innovation Lab pavilion, and are rolling out robots both in houses and in aged care facilities. There are two units pictured and the height of the platform can raise or lower depending on whether it needs to reach a countertop or fridge unit to retrieve items, like drinks and medications, or whether it needs to lower to become a bed or chair side table. These units can be commanded by voice, or tablet, or scheduled to travel around designated ‘bus stops’, using advanced localization and mapping. The team at Labrador have a wealth of experience at other consumer robotics companies.

Two Retrievers from Labrador Robotics in the AARP Innovation Lab Pavilion at CES 2023. (Image source: Andra Keay)

I first met Sampriti Battacharya, pictured below with her autonomous electric boat, when she was still doing her robotics PhD at MIT, dreaming about starting her own company. Five short years later, she’s now the proud founder of Navier with not one but two working prototypes of the ‘boat of the future’. The Navier 30 is a 30’ long electric intelligent hydrofoil with a range of 75 nautical miles and a top speed of 35 knots. Not only is the electric hydrofoil 90% more energy efficient than traditional boats but it eliminates sea sickness with a super smooth ride. Sampriti’s planning to revolutionize public transport for cities that span waterways, like San Francisco or Boston or New York.

Navier’s ‘boat of the future’ with founder Sampriti Battacharya, plus an extra stowaway quadruped robot from Unitree. Image source: Andra Keay

Another rapidly evolving robotics company is Yarbo. Starting out as a prototype snow blowing robot, after five years of solid R&D, Snowbot has evolved into the Yarbo modular family of smart yard tools. Imagine a smart docking mobile base which can be turned from a lawn mower to a snow blower or a leaf blower. It can navigate autonomously, and it’s electric of course.

And these robotics companies are making waves at CES. I met French startup Acwa Robotics earlier in 2022 and was so impressed that I nominated them as an IEEE Entrepreneurship Star. Water utilities around the world are faced with aging and damaged infrastructure, inaccessible underground pipes, responsible for huge amounts of water loss, road and building damage. Acwa’s intelligent robot travels inside the pipes without stopping water flow and provides rapid precisely localized inspection data that can pinpoint leaks, cracks and deterioration. Acwa was nominated for honors in the Smart Cities category and also won CES Best of Innovation Award.

Acwa Robotics and CES 2023 Best of Innovation Awards (Image Source: Acwa Robotics)

Some other robotics companies and startups worth looking at were Apex.ai, Caterpillar, Unitree, Bosch Rexroth, Waymo, Zoox, Whill, Meropy, Artemis Robotics, Fluent Pet and Orangewood. Let me know who I missed! According to the app, 278 companies tagged themselves as Robotics, 73 as Drones, 514 as Vehicle Tech, and 722 as Startups, although I’d say the overall number of exhibitors and attendees was down on previous years although there were definitely more robots.

View Details

Countries around the world invest in robotics to support developments in industry and society. What are the exact targets of robotics research funding programs (R&D) officially driven by governments in Asia, Europe and America today? This has been researched by the International Federation of Robotics and published in the 2023 update paper of “World Robotics R&D Programs”.

© Pixabay

“The 3rd version of World Robotics R&D Programs covers the latest funding developments including updates in 2022,” says Prof. Dr. Jong-Oh Park, Vice-Chairman IFR Research Committee and member of the Executive Board.

The overview shows that the most advanced robotics countries in terms of annual installations of industrial robots – China, Japan, USA, South Korea, Germany – and the EU drive very different R&D strategies:

Robotics R&D programs – officially driven by governmentsIn China, the “14th Five-Year Plan” for Robot Industry Development, released by the Ministry of Industry and Information Technology (MIIT) in Beijing on 21st December 2021, focuses on promoting innovation. The goal is to make China a global leader for robot technology and industrial advancement. Robotics is included in 8 key industries for the next 5 years. In order to implement national science and technology innovation arrangements, the key special program “Intelligent Robots” was launched under the National Key R&D Plan on 23rd April 2022 with a funding of 43.5 million USD. The recent statistical yearbook “World Robotics” by IFR shows that China reached a robot density of 322 units per 10,000 workers in the manufacturing industry: The country ranks 5th worldwide in 2021 compared to 20th (140 units) in 2018.

In Japan, the “New Robot Strategy” aims to make the country the world´s number one robot innovation hub. More than 930.5 million USD in support has been provided by the Japanese government in 2022. Key sectors are manufacturing (77.8 million USD), nursing and medical (55 million USD), infrastructure (643.2 million USD) and agriculture (66.2 million USD). The action plan for manufacturing and service includes projects such as autonomous driving, advanced air mobility or the development of integrated technologies that will be the core of next-generation artificial intelligence and robots. A budget of 440 million USD was allocated to robotics-related projects in the “Moonshot Research and Development Program” over a period of 5 years from 2020 to 2025. According to the statistical yearbook “World Robotics” by IFR, Japan is the world´s number one industrial robot manufacturer and delivered 45% of the global supply in 2021.

The 3rd Basic Plan on Intelligent Robots of South Korea is pushing to develop robotics as a core industry in the fourth industrial revolution. The Korean government allocated 172.2 million USD in funding for the “2022 Implementation Plan for the Intelligent Robot”. From 2022 to 2024 a total of 7.41 million USD is planned in funding for the “Full-Scale Test Platform Project for Special-Purpose Manned or Unmanned Aerial Vehicles”. The statistical yearbook “World Robotics” showed an all-time high of 1,000 industrial robots per 10,000 employees in 2021. This makes Korea the country with the highest robot density worldwide.

Horizon Europe is the European Union’s key research and innovation framework program with a budget of 94.30 billion USD for seven years (2021-2027). Top targets are: strengthening the EU’s scientific and technological bases, boosting Europe’s innovation capacity, competitiveness and jobs as well as delivering on citizens’ priorities and sustaining socio-economic models and values. The European Commission provides total funding of 198.5 million USD for the robotics-related work program 2021-2022.

Germany´s High-Tech Strategy 2025 (HTS) is the fourth edition of the German R&D and innovation program. The German government will provide around 69 million USD annually until 2026 – a total budget of 345 million USD for five years. As part of the HTS 2025 mission, the program “Shaping technology for the people” was launched. This program aims to use technological change in society as a whole and in the world of work for the benefit of people. Research topics are: digital assistance systems such as data glasses, human-robot-collaboration, exoskeletons to support employees in their physical work, but also solutions for the more flexible organization of work processes or the support of mobile work. According to the report “World Robotics” by IFR, Germany is the largest robot market in Europe – the robot density ranks in 4th place worldwide with 397 units per 10,000 employees.

The National Robotics Initiative (NRI) in the USA was launched for fundamental robotics R&D supported by the US government. The NRI-3.0 program, announced in February 2021, seeks research on integrated robot systems and builds upon the previous NRI programs. The US government supported the NRI-3.0 fund to the sum of 14 million USD in 2021. Collaboration among academics, industry, government, non-profit, and other organizations is encouraged. The “Moon to Mars” project by NASA for example highlights objectives to establish a long-term presence in the vicinity of and on the moon. The projects target research and technology development that will significantly increase the performance of robots to collaboratively support deep space human exploration and science missions. For the Artemis lunar program, the US government is planning to allocate a budget of 35 billion USD from 2020 to 2024. The statistical yearbook “World Robotics” by IFR shows that robot density in the United States rose from 255 units in 2020 to 274 units in 2021. The country ranks 9th in the world. Regarding annual installations of industrial robots, the USA takes 3rd position.

View Details

Claire chatted to Professor Dan Stoyanov from University College London all about robotic vision, surgical robotics, and artificial intelligence.

Dan Stoyanov, FREng, FIET, is a Professor at UCL Computer Science holding a Royal Academy of Engineering Chair in Emerging Technologies. He is Director of the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), a large research centre combining engineering and clinical expertise. His research interests are focused on surgical robotics, surgical data science and the development of surgical AI systems for clinical use. He is Chief Scientist at Digital Surgery, and he co-founded Odin Vision Ltd as a UCL spin-out focused on AI in gastroenterology.

View Details

Robot developments and the study of social processes can happen side-by-side in RoboHouse. Because we feel that technology should learn to look beyond its own horizons, if we aim to make the workplace more attractive. Why are people leaving the jobs they used to love? What’s going on in crucial sectors like healthcare, agriculture and manufacturing?

To explore these questions, we go into the field with scientists and innovators. Under the banner of FRAIM, our new transdisciplinary research centre dedicated to the future of work. What do robot specialists notice when they travel to places where people and robots work together?

Our latest instalment of FRAIM in the Field follows Maria Luce Lupetti as she meets with Henk Verdegaal on a grey November day. Last August Verdegaal, flower bulb farmer in the Netherlands, finally saw what an agricultural robot could do on his lands. The Agbot by developer AgXeed was humming along, managed by a “smart and ready to use autonomy system with a full suite of vehicle peripherals.”

As the Agbot demonstration progressed, flower bulb farmer Henk Verdegaal became more and more convinced of the potential of this particular way of implementing robotics: “I feel this could be deployed for all things related to light ground work. The only issue is: can the system run for enough hours to make it worthwhile financially. That seems hard at the moment.”Henk Verdegaal experiments with smart technology to reduce his use of pesticides. He expects that systems like Agbot can also reduce his reliance on labour and liberate him from field work, so that he can focus on more important processes. Drones however, have so far failed to impress Verdegaal. Connectivity issues caused the drone to lose its way and communicate poorly with the camera.

Assistant professor Maria Luce Lupetti, specialised in critical design for AI systems at TU Delft, arrives at a sobering insight during FRAIM in the Field: “In a place like a farm there are clear problems like not finding people to drive the truck. So it automatically makes you think: ‘OK, you make it autonomous. You have a clear need for that, the technology is there.’ But there are reasons why people have a hard time finding workers. These problems are systemic. There are financial issues, there are sustainability issues. There is a pressing housing crisis that makes the price of the land rise. A lot of different forces are coming together to influence the work of people on a farm.”

Watch the rest of the series here, or on our youtube channel.

The post When a professor meets a farmer appeared first on RoboHouse.

View Details

Researchers are ushering in a new way of thinking about robots in the workplace based on the idea of robots and workers as teammates rather than competitors. © BigBlueStudio, Shutterstock

For decades, the arrival of robots in the workplace has been a source of public anxiety over fears that they will replace workers and create unemployment.

Now that more sophisticated and humanoid robots are actually emerging, the picture is changing, with some seeing robots as promising teammates rather than unwelcome competitors.

‘Cobot’ colleaguesTake Italian industrial-automation company Comau. It has developed a robot that can collaborate with – and enhance the safety of – workers in strict cleanroom settings in the pharmaceutical, cosmetics, electronics, food and beverage industries. The innovation is known as a “collaborative robot”, or “cobot”.

Comau’s arm-like cobot, which is designed for handling and assembly tasks, can automatically switch from an industrial to a slower speed when a person enters the work area. This new feature allows one robot to be used instead of two, maximising productivity and protecting workers.

‘It has advanced things by allowing a dual mode of operation,’ said Dr Sotiris Makris, a roboticist at the University of Patras in Greece. ‘You can either use it as a conventional robot or, when it is in collaborative mode, the worker can grab it and move it around as an assisting device.’

Makris was coordinator of the just-completed EU-funded SHERLOCK project, which explored new methods for safely combining human and robot capabilities from what it regarded as an often overlooked research angle: psychological and social well-being.

Creative and inclusiveRobotics can help society by carrying out repetitive, tedious tasks, freeing up workers to engage in more creative activities. And robotic technologies that can collaborate effectively with workers could make workplaces more inclusive, such as by aiding people with disabilities.

“There is increasing competition around the globe, with new advances in robotics.”

– Dr Sotiris Makris, SHERLOCK

These opportunities are important to seize as the structure and the age profile of the European workforce changes. For example, the proportion of 55-to-64-year-olds increased from 12.5% of the EU’s employees in 2009 to 19% in 2021.

Alongside the social dimension, there is also economic benefit from greater industrial efficiency, showing that neither necessarily needs to come at the expense of the other.

‘There is increasing competition around the globe, with new advances in robotics,’ said Makris. ‘That is calling for actions and continuous improvement in Europe.’

Makris cites the humanoid robots being developed by Elon Musk-led car manufacturer Tesla. Wearable robotics, bionic limbs and exoskeleton suits are also being developed that promise to enhance people’s capabilities in the workplace.

Still, the rapidly advancing wave of robotics poses big challenges when it comes to ensuring they are effectively integrated into the workplace and that people’s individual needs are met when working with them.

Case for SHERLOCKSHERLOCK also examined the potential for smart exoskeletons to support workers in carrying and handling heavy parts at places such as workshops, warehouses or assembly sites. Wearable sensors and AI were used to monitor and track human movements.

With this feedback, the idea is that the exoskeleton can then adapt to the needs of the specific task while helping workers retain an ergonomic posture to avoid injury.

‘Using sensors to collect data from how the exoskeleton performs allowed us to see and better understand the human condition,’ said Dr Makris. ‘This allowed us to have prototypes on how exoskeletons need to be further redesigned and developed in the future, depending on different user profiles and different countries.’

SHERLOCK, which has just ended after four years, brought together 18 European organisations in multiple countries from Greece to Italy and the UK working on different areas of robotics.

The range of participants enabled the project to harness a wide variety of perspectives, which Dr Makris said was also beneficial in the light of differing national rules on integrating robotics technology.

As a result of the interaction of these robotic systems with people, the software is advanced enough to give direction to ‘future developments on the types of features to have and how the workplace should be designed,’ said Dr Makris.

Old hands, new toolsAnother EU-funded project that ended this year, CO-ADAPT, used cobots to help older people navigate the digitalised workplace.

“You find interesting differences in how much the machine and how much the person should do.”

– Prof Giulio Jacucci, CO-ADAPT

The project team developed a cobot-equipped adaptive workstation to aid people in assembly tasks, such as making a phone, car or toy – or, indeed, combining any set of individual components into a finished product during manufacturing. The station can adapt workbench height and lighting to a person’s physical characteristics and visual abilities. It also includes features like eye-tracking glasses to gather information on mental workload.

That brings more insight into what all kinds of people need, said Professor Giulio Jacucci, coordinator of CO-ADAPT and a computer scientist at the University of Helsinki in Finland.

‘You find interesting differences in how much the machine and how much the person should do, as well as how much the machine should try to give guidance and how,’ Jacucci said. ‘This is important work that goes down to the nuts and bolts of making this work.’

Still, cobot-equipped workplaces that can fully tap into and respond to people’s mental states in real-life settings could still be a number of years away, he said.

‘It’s so complex because there’s the whole mechanical part, plus trying to understand people’s status from their psychophysiological states,’ said Prof Jacucci.

Meanwhile, because new technologies can be used in much simpler ways to improve the workplace, CO-ADAPT also explored digitalisation more broadly.

Smart shiftsOne area was software that enables ‘smart-shift scheduling’, which arranges duty periods for workers based on their personal circumstances. The approach has been shown to reduce sick leave, stress and sleep disorders among social welfare and health care workers.

‘It’s a fantastic example of how workability improves because we use evidence-based knowledge of how to have well-being-informed schedules,’ said Prof Jacucci.

Focusing on the individual is key to the future of well-integrated digital tools and robotics, he said.

‘Let’s say you have to collaborate with some robot in an assembly task,’ he said. ‘The question is: should the robot be aware of my cognitive and other abilities? And how should we divide the task between the two?’

The basic message from the project is that plenty of room exists to improve and broaden working environments.

‘It shows how much untapped potential there is,’ said Prof Jacucci.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

View Details

Prof. Hae-Won Park (left), Ph.D. Student Yong Um (centre), Ph.D. Student Seungwoo Hong (right). Credits: KAIST

We had the chance to interview Hae-Won Park, Seungwoo Hong and Yong Um, authors of the paper “Agile and versatile climbing on ferromagnetic surfaces with a quadrupedal robot”, recently published in Science Robotics.

What is the topic of the research in your paper?
The main topic of our work is that the robot we have developed can move agilely, not only on flat ground but also on vertical walls and ceilings made of ferromagnetic materials. Also, it has the ability to perform dexterous maneuvers such as crossing gaps, overcoming obstacles, and transitioning upon corners.

Could you tell us about the implications of your research and why it is an interesting area for study?
Such agile and dexterous locomotion capabilities will be able to expand the robot’s operational workspace and approach places that are difficult or dangerous for human operators to access directly. For example, inspection and welding operations in heavy industries such as shipbuilding, steel bridges, and storage tanks.

Could you explain your methodology? What were your main findings?
Our magnet foot can switch the on/off state in a short period of time (5 ms) and in an energy-efficient way, thanks to the novel geometry design of EPM. At the same time, the magnet foot can provide large holding forces in both shear and normal directions due to the MRE footpad. Also, our actuators can provide balanced speed/torque characteristics, high-bandwidth torque control capability, and the ability to mediate high impulsive force. To control vertical and inverted locomotion as well as various versatile motions, we have utilized a control framework (model predictive control) that can generate reliable and robust reaction forces to track desired body motions in 3D space while preventing slippage or tipping-over occurs. We found that all the elements mentioned earlier are imperative to perform dynamic maneuvers against gravity.

What further work are you planning in this area?
So far, the robot is able to move on smooth surfaces with moderate curvature. To enable the robot to move on irregularly shaped surfaces, we are working on designing a compliantly-integrated multiple miniaturized EPMs with MRE footpads that can increase the effective contact area to provide robust adhesion. Also, a vision system with high-level navigation algorithms will be included to enable the robot to move autonomously in the near future.

About the authors

| | Hae-Won Park received the B.S. and M.S. degrees from Yonsei University, Seoul, South Korea, in 2005 and 2007, respectively, and the Ph.D. degree from the University of Michigan, Ann Arbor, MI, USA, in 2012, all in mechanical engineering. He is an Associate Professor of mechanical engineering with the Korea Advanced Institute of Science and Technology, Daejeon, South Korea. His research interests include the intersection of control, dynamics, and mechanical design of robotic systems, with special emphasis on legged locomotion robots. Dr. Park is the recipient of the 2018 National Science Foundation (NSF) CAREER Award and NSF most prestigious awards in support of early-career faculty. |

| | Seungwoo Hong received the B.S. degree from Shanghai Jiao Tong University, Shanghai, China, in July 2014, and the M.S. degree from Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in August 2017, all in mechanical engineering. He is currently a Ph.D. candidate with the Department of Mechanical Engineering, KAIST, Daejeon, Korea. His current research interests include model-based optimization, motion planning and control of legged robotic systems. |

| | Yong Um received the B.S. degree in mechanical engineering from the Korea Advanced Institute of Science and Technology, Daejeon, South Korea, in 2020. He is currently working toward the Ph.D. degree in mechanical engineering in Korea Advanced Institute of Science and Technology. His research interests include mechanical system and magnetic device design for legged robot. |

• PAPER – Agile and versatile climbing on ferromagnetic surfaces with a quadrupedal robot. Seungwoo Hong, Yong Um, Jaejun Park, and Hae-Won Park. Science Robotics 7.73 (2022): eadd1017.

View Details

CES Unveiled was jam packed with the latest and greatest tech from companies from all over the world. Get a behind-the-scenes look with Brian Tong at the innovations we saw.

C Space Studio & Anchor Desk interviewsKeynotes & Insider look

View Details

Claire chatted to Mollie Claypool from Automated Architecture about robot house-building, zero-carbon architecture, and community participation.

Mollie Claypool is CEO of AUAR Ltd, a tech company revolutionising house building using automation. Mollie is a leading architecture theorist focused on issues of social justice highlighted by increasing automation in architecture and design production. She is also Associate Professor in Architecture at The Bartlett School of Architecture, UCL.

View Details

The EPSRC UK Robotics and Autonomous Systems (UK-RAS) Network is pleased to announce the official launch of its 2023 competitions, inviting the UK’s primary schoolchildren to share their creative robot designs and imaginative stories with a panel of experts, for a chance to win some unique prizes. These annual competitions, which have proved hugely popular with budding authors and illustrators nationwide, are now returning for the fourth year.

The “Draw A Robot” competition challenges children in Key Stage 1 (aged 5-7 years old) to design a robot that they’d like to see in the future. Children can use whichever drawing materials they prefer — paper, pens, pencils, paints, crayons, or even natural materials — to create their ideal robot design, and the robot can be designed to perform any task or job. Competition participants will be able to explain their robot’s functions by labelling gadgets and features on the drawing and writing a short design spec.

For the “Once Upon A Robot” writing competition, Key Stage 2 children (aged 7-11 years old), are invited to write an imaginative short story featuring any kind of robot – or robots – their imagination can conjure! Children will have up to 800 words to tell their creative robot tales and they can choose any literary genre they like. It could be a spine-tingling horror, an action-packed adventure, or even a light-hearted comedy.

ZOOG by Matilde Facchini, age 7 (Draw a Robot 2022 Winner)

Plan-o-bot by Tehmina Walker, age 6 (Draw a Robot 2021 Winner)

The two competitions will be judged by robotics experts from the organising ESPRC UK-RAS Network, plus two very special invited judges. The writing competition will be judged this year by award-winning author Sharna Jackson, whose inspiring and mystifying books include High-Rise Mystery and The Good Turn. The drawing competition will be judged by internationally acclaimed Anglo/American author, illustrator and artist Ted Dewan, creator of the Emmy-Award-winning animated television series Bing.

This year’s exclusive prize packages include:

Draw A Robot Competition winner

• Thames & Kosmos Coding and Robotics kit – contributed by competition partner the University of Sheffield Advanced Manufacturing Research Centre (AMRC)
• A tour of the AMRC’s Factory 2050 in Sheffield, the UK’s first state-of-the-art factory dedicated to conducting collaborative research into reconfigurable digitally assisted assembly, component manufacturing and machining technologies
• A copy of the book “The Sorcerer’s Apprentice”, signed by competition judge Ted Dewan

Draw A Robot Competition runner-up

• 4M Green Science Solar Hybrid Power Aqua Robot – contributed by competition partner the UKRI Trust Worthy Autonomous Systems (TAS) hub
• A copy of the book “Top Secret”, signed by competition judge Ted Dewan

Once Upon A Robot Competition winner

• Lego Mindstorms Robot Inventor kit – contributed by competition partner Birmingham Extreme Robotics Lab
• A tour of the Extreme Robotics Lab in Birmingham and a robotics masterclass from RobotCoders for the winner and a friend
• Printed copy of the winning story with bespoke illustrations by illustrator and science communicator Hana Ayoob
• A copy of the books “The Good Turn” and “Black Artists Shaping the World”, signed by competition judge Sharna Jackson

Once Upon A Robot Competition runner-up

• Maqueen Lite – micro:bit – contributed by competition partner The National Robotarium
• A copy of the book “High-Rise Mystery”, signed by competition judge Sharna Jackson

For more information, details of prizes, judging criteria and to submit an entry, please visit https://www.ukras.org.uk/school-robot-competition/.

Both competitions are open for entry from the 10th January and will close for submissions on the 23rd April. The winners will be announced at a special virtual award ceremony due to be held on 22nd June 2023.

EPSRC UK-RAS Network Chair Prof. Robert Richardson says: “We are absolutely delighted to be launching these two fantastic competitions for primary schoolchildren for the fourth year running, which offer the next generation a creative way to engage with the exciting world of robotics and automation. We can’t wait to see the imagination and ingenuity that the nation’s young authors and artists bring to these challenges, and we look forward to the very enjoyable task of judging this year’s entries.”

The two creative competitions for young children were first launched in 2020 for UK Robotics Week, now the UK Festival of Robotics – a 7-day celebration of robotics and intelligent systems held at the end of June. This annual celebration is hosted by the EPSRC UK Robotics and Autonomous Systems (UK-RAS) Network, which provides academic leadership in robotics and coordinates activities at 35 partner universities across the UK.

View Details

Here’s my annual summary of the top stories of the prior year. This time the news was a strong mix of bad and good.

Read the text story on Forbes.com at Robocars 2022 year in review.

And see the video version here:

View Details

Artificial intelligence is set to revolutionise agriculture by helping farmers meet field-hand needs and identify diseased plants. © baranozdemir, iStock

In the Dutch province of Zeeland, a robot moves swiftly through a field of crops including sunflowers, shallots and onions. The machine weeds autonomously – and tirelessly – day in, day out.

“Farmdroid” has made life a lot easier for Mark Buijze, who runs a biological farm with 50 cows and 15 hectares of land. Buijze is one of the very few owners of robots in European agriculture.

Robots to the rescueHis electronic field worker uses GPS and is multifunctional, switching between weeding and seeding. With the push of a button, all Buijze has to do is enter coordinates and Farmdroid takes it from there.

‘With the robot, the weeding can be finished within one to two days – a task that would normally take weeks and roughly four to five workers if done by hand,’ he said. ‘By using GPS, the machine can identify the exact location of where it has to go in the field.’

About 12 000 years ago, the end of foraging and start of agriculture heralded big improvements in people’s quality of life. Few sectors have a history as rich as that of farming, which has evolved over the centuries in step with technological advancements.

In the current era, however, agriculture has been slower than other industries to follow one tech trend: artificial intelligence (AI). While already commonly used in forms ranging from automated chatbots and face recognition to car braking and warehouse controls, AI for agriculture is still in the early stages of development.

Now, advances in research are spurring farmers to embrace robots by showing how they can do everything from meeting field-hand needs to detecting crop diseases early.

Lean and greenFor French agronomist Bertrand Pinel, farming in Europe will require far greater use of robots to be productive, competitive and green – three top EU goals for a sector whose output is worth around €190 billion a year.

“Labour is one of the biggest obstacles in agriculture.”

– Fritz van Evert, ROBS4CROPS

One reason for using robots is the need to forgo the use of herbicides by eliminating weeds the old-fashioned way: mechanical weeding, a task that is not just mundane but also arduous and time consuming. Another is the frequent shortage of workers to prune grapevines.

‘In both cases, robots would help,’ said Pinel, who is research and development project manager at France-based Terrena Innovation. ‘That is our idea of the future for European agriculture.’

Pinel is part of the EU-funded ROBS4CROPS project. With some 50 experts and 16 institutional partners involved, it is pioneering a robot technology on participating farms in the Netherlands, Greece, Spain and France.

‘This initiative is quite innovative,’ said Frits van Evert, coordinator of the project. ‘It has not been done before.’

In the weeds AI in agriculture looks promising for tasks that need to be repeated throughout the year such as weeding, according to van Evert, a senior researcher in precision agriculture at Wageningen University in the Netherlands.

‘If you grow a crop like potatoes, typically you plant the crop once per year in the spring and you harvest in the fall, but the weeding has to be done somewhere between six and 10 times per year,’ he said.

Plus, there is the question of speed. Often machines work faster than any human being can.

“With this robot everything is done in the field.”

– Francisco Javier Nieto De Santos, FLEXIGROBOTS

Francisco Javier Nieto De Santos, coordinator of the EU-funded FLEXIGROBOTS project, is particularly impressed by a model robot that takes soil samples. When done by hand, this practice requires special care to avoid contamination, delivery to a laboratory and days of analysis.

‘With this robot everything is done in the field,’ De Santos said. ‘It can take several samples per hour, providing results within a matter of minutes.’

Eventually, he said, the benefits of such technologies will extend beyond the farm industry to reach the general public by increasing the overall supply of food.

Unloved labour Meanwhile, agricultural robots may be in demand not because they can work faster than any person but simply because no people are available for the job.

Even before inflation rates and fertiliser prices began to surge in 2021 amid an energy squeeze made worse by Russia’s invasion of Ukraine this year, farmers across Europe were struggling on another front: finding enough field hands including seasonal workers.

‘Labour is one of the biggest obstacles in agriculture,’ said van Evert. ‘It’s costly and hard to get these days because fewer and fewer people are willing to work in agriculture. We think that robots, such as self-driving tractors, can take away this obstacle.’

The idea behind ROBS4CROPS is to create a robotic system where existing agricultural machinery is upgraded so it can work in tandem with farm robots.

For the system to work, raw data such as images or videos must first be labelled by researchers in ways than can later be read by the AI.

Driverless tractorsThe system then uses these large amounts of information to make “smart” decisions as well as predictions – think about the autocorrect feature on laptop computers and mobile phones, for example.

A farming controller comparable to the “brain” of the whole operation decides what needs to happen next or how much work remains to be done and where – based on information from maps or instructions provided by the farmer.

The machinery – self-driving tractors and smart implements like weeders equipped with sensors and cameras – gathers and stores more information as it works, becoming “smarter”.

Crop protectionFLEXIGROBOTS, based in Spain, aims to help farmers use existing robots for multiple tasks including disease detection.

Take drones, for example. Because they can spot a diseased plant from the air, drones can help farmers detect sick crops early and prevent a wider infestation.

‘If you can’t detect diseases in an early stage, you may lose the produce of an entire field, the production of an entire year,’ said De Santos. ‘The only option is to remove the infected plant.’

For example, there is no treatment for the fungus known as mildew, so identifying and removing diseased plants early on is crucial.

Pooling information is key to making the whole system smarter, De Santos said. Sharing data gathered by drones with robots or feeding the information into models expands the “intelligence” of the machines.

Although agronomist Pinel doesn’t believe that agriculture will ever be solely reliant on robotics, he’s certain about their revolutionary impact.

‘In the future, we hope that the farmers can just put a couple of small robots in the field and let them work all day,’ he said.

Research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

Watch the video

This article was originally published in Horizon, the EU Research and Innovation magazine.

View Details

Joey’s design. Image credit: TL Nguyen, A Blight, A Pickering, A Barber, GH Jackson-Mills, JH Boyle, R Richardson, M Dogar, N Cohen

By Mischa Dijkstra, Frontiers science writer

Researchers from the University of Leeds have developed the first mini-robot, called Joey, that can find its own way independently through networks of narrow pipes underground, to inspect any damage or leaks. Joeys are cheap to produce, smart, small, and light, and can move through pipes inclined at a slope or over slippery or muddy sediment at the bottom of the pipes. Future versions of Joey will operate in swarms, with their mobile base on a larger ‘mother’ robot Kanga, which will be equipped with arms and tools for repairs to the pipes.

Beneath our streets lies a maze of pipes, conduits for water, sewage, and gas. Regular inspection of these pipes for leaks, or repair, normally requires these to be dug up. The latter is not only onerous and expensive – with an estimated annual cost of £5.5bn in the UK alone – but causes disruption to traffic as well as nuisance to people living nearby, not to mention damage to the environment.

Now imagine a robot that can find its way through the narrowest of pipe networks and relay images of damage or obstructions to human operators. This isn’t a pipedream anymore, shows a study in Frontiers in Robotics and AI by a team of researchers from the University of Leeds.

“Here we present Joey – a new miniature robot – and show that Joeys can explore real pipe networks completely on their own, without even needing a camera to navigate,” said Dr Netta Cohen, a professor at the University of Leeds and the final author on the study.

Joey is the first to be able to navigate all by itself through mazes of pipes as narrow as 7.5 cm across. Weighing just 70 g, it’s small enough to fit in the palm of your hand.

Pipebots projectThe present work forms part of the ‘Pipebots’ project of the universities of Sheffield, Bristol, Birmingham, and Leeds, in collaboration with UK utility companies and other international academic and industrial partners.

First author Dr Thanh Luan Nguyen, a postdoctoral scientist at the University of Leeds who developed Joey’s control algorithms (or ‘brain’), said: “Underground water and sewer networks are some of the least hospitable environments, not only for humans, but also for robots. Sat Nav is not accessible undergound. And Joeys are tiny, so have to function with very simple motors, sensors, and computers that take little space, while the small batteries must be able to operate for long enough.”

Joey moves on 3D-printed ‘wheel-legs’ that roll through straight sections and walk over small obstacles. It is equipped with a range of energy-efficient sensors that measure its distance to walls, junctions, and corners, navigational tools, a microphone, and a camera and ‘spot lights’ to film faults in the pipe network and save the images. The prototype cost only £300 to produce.

Mud and slippery slopesThe team showed that Joey is able to find its way, without any instructions from human operators, through an experimental network of pipes including a T-junction, a left and right corner, a dead-end, an obstacle, and three straight sections. On average, Joey managed to explore about one meter of pipe network in just over 45 seconds.

To make life more difficult for the robot, the researchers verified that the robot easily moves up and down inclined pipes with realistic slopes. And to test Joey’s ability to navigate through muddy or slippery tubes, they also added sand and gooey gel (actually dishwashing liquid) to the pipes – again with success.

Importantly, the sensors are enough to allow Joey to navigate without the need to turn on the camera or use power-hungry computer vision. This saves energy and extends Joey’s current battery life. Whenever the battery runs low, Joey will return to its point of origin, to ‘feed’ on power.

Currently, Joeys have one weakness: they can’t right themselves if they inadvertently turn on their back, like an upside-down tortoise. The authors suggest that the next prototype will be able to overcome this challenge. Future generations of Joey should also be waterproof, to operate underwater in pipes entirely filled with liquid.

Joey’s future is collaborativeThe Pipebots scientists aim to develop a swarm of Joeys that communicate and work together, based off a larger ‘mother’ robot named Kanga. Kanga, currently under development and testing by some of the same authors at Leeds School of Computing, will be equipped with more sophisticated sensors and repair tools such as robot arms, and carry multiple Joeys.

“Ultimately we hope to design a system that can inspect and map the condition of extensive pipe networks, monitor the pipes over time, and even execute some maintenance and repair tasks,” said Cohen.

“We envision the technology to scale up and diversify, creating an ecology of multi-species of robots that collaborate underground. In this scenario, groups of Joeys would be deployed by larger robots that have more power and capabilities but are restricted to the larger pipes. Meeting this challenge will require more research, development, and testing over 10 to 20 years. It may start to come into play around 2040 or 2050.”

Top half: navigating through a T-junction in the pipe network. Bottom half: encountering an obstruction and turning back. Image credit: TL Nguyen, A Blight, A Pickering, A Barber, GH Jackson-Mills, JH Boyle, R Richardson, M Dogar, N Cohen

Top half: moving through sand, slippery goo, or mud. Bottom half: moving through pipe sloped at an angle. Image credit: TL Nguyen, A Blight, A Pickering, A Barber, GH Jackson-Mills, JH Boyle, R Richardson, M Dogar, N Cohen

• PAPER – Autonomous control for miniaturized mobile robots in unknown pipe networks. T. L. Nguyen, A. Blight, A. Pickering, G. Jackson-Mills, A. R. Barber, J. H. Boyle, R. Richardson, M. Dogar, and N. Cohen. Frontiers in Robotics and AI, 309.

View Details

Image generated by DALLE 2 using prompt “a hyperrealistic image of a robot watching robot videos on a laptop”

Did you manage to watch all the holiday robot videos of 2022? If you did but are still hungry for more, I have a few more videos from Science Magazine featuring robotics research that were released during last year. Enjoy!

Extra: breakthrough of the year

View Details

elwynn/Shutterstock

By Paul Cureton (Senior Lecturer in Design (People, Places, Products), Lancaster University) and Ole B. Jensen (Professor of Urban Theory and Urban Design, Aalborg University)

Drones are already shaping the face of our cities – used for building planning, heritage, construction and safety enhancement. But, as studies by the UK’s Department of Transport have found, swathes of the public have a limited understanding of how drones might be practically applied.

It’s crucial that the ways drones are affecting our future are understood by the majority of people. As experts in design futures and mobility, we hope this short overview of five ways drones will affect building design offers some knowledge of how things are likely to change.

Infographic showcasing other ways drones will influence future building design. Nuri Kwon, Drone Near-Futures, Imagination Lancaster, Author provided

1. Creating digital models of buildingsDrones can take photographs of buildings, which are then used to build 3D models of buildings in computer-aided design software.

These models have accuracy to within a centimetre, and can be combined with other data, such as 3D scans of interiors using drones or laser scanners, in order to provide a completely accurate picture of the structure for surveyors, architects and clients.

Using these digital models saves time and money in the construction process by providing a single source thaOle B. Jensent architects and planners can view.

1. Heritage simulationsStudio Drift are a multidisciplinary team of Dutch artists who have used drones to construct images through theatrical outdoor drone performances at damaged national heritage sites such as the Notre Dame in Paris, Colosseum in Rome and Gaudí’s Sagrada Familia in Barcelona.

Drones could be used in the near-future in a similar way to help planners to visualise the final impact of restoration or construction work on a damaged or partially finished building.

1. Drone deliveryThe arrival of drone delivery services will see significant changes to buildings in our communities, which will need to provide for docking stations at community hubs, shops and pick-up points.

Wingcopter are one of many companies trialling delivery drones. Akash 1997, CC BY-SA

There are likely to be landing pads installed on the roofs of residential homes and dedicated drone-delivery hubs. Research has shown that drones can help with the last mile of any delivery in the UK, Germany, France and Italy.

Architects of the future will need to add these facilities into their building designs.

1. Drones mounted with 3D printersTwo research projects from architecture, design, planning, and consulting firm Gensler and another from a consortium led by Imperial College London (comprising University College London, University of Bath, University of Pennsylvania, Queen Mary University of London, and Technical University of Munich) named Empa have been experimenting with drones with mounted 3D printers. These drones would work at speed to construct emergency shelters or repair buildings at significant heights, without the need for scaffolding, or in difficult to reach locations, providing safety benefits.

Gensler have already used drones for wind turbine repair and researchers at Imperial College are exploring bee-like drone swarms that work together to construct blueprints. The drones coordinate with each other to follow a pre-defined path in a project called Aerial Additive Manufacturing. For now, the work is merely a demonstration of the technology, and not working on a specific building.

In the future, drones with mounted 3D printers could help create highly customised buildings at speed, but how this could change the workforce and the potential consequences for manual labour jobs is yet to be understood.

1. Agile surveillanceDrones offer new possibilities for surveillance away from the static, fixed nature of current systems such as closed circuit television.

Drones with cameras and sensors relying on complex software systems such as biometric indicators and “face recognition” will probably be the next level of surveillance applied by governments and police forces, as well as providing security monitoring for homeowners. Drones would likely be fitted with monitoring devices, which could communicate with security or police forces.

Drones used in this way could help our buildings become more responsive to intrusions, and adaptable to changing climates. Drones may move parts of the building such as shade-creating devices, following the path of the sun to stop buildings overheating, for example.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

View Details

Episode 24 – Gopal RamchurnClaire chatted to Gopal Ramchurn from the University of Southampton about artificial intelligence, autonomous systems and renewable energy.

Sarvapali (Gopal) Ramchurn is a Professor of Artificial Intelligence, Turing Fellow, and Fellow of the Institution of Engineering and Technology. He is the Director of the UKRI Trustworthy Autonomous Systems hub and Co-Director of the Shell-Southampton Centre for Maritime Futures. He is also a Co-CEO of Empati Ltd, an AI startup working on decentralised green hydrogen technologies. His research is about the design of Responsible Artificial Intelligence for socio-technical applications including energy systems and disaster management.

Episode 25 – Ferdinando Rodriguez y BaenaClaire chatted to Ferdinando Rodriguez y Baena from Imperial College London about medical robotics, robotic surgery, and translational research.

Ferdinando Rodriguez y Baena is Professor of Medical Robotics in the Department of Mechanical Engineering at Imperial College, where he leads the Mechatronics in Medicine Laboratory and the Applied Mechanics Division. He has been the Engineering Co-Director of the Hamlyn Centre, which is part of the Institute of Global Health Innovation, since July 2020. He is a founding member and great advocate of the Imperial College Robotics Forum, now the first point of contact for roboticists at Imperial College.

Episode 26 – Séverin LemaignanClaire chatted to Séverin Lemaignan from PAL Robotics all about social robots, behaviour, and robot-assisted human-human interactions.

Séverin Lemaignan is Senior Scientist at Barcelona-based PAL Robotics. He leads the Social Intelligence team, in charge of designing and developing the socio-cognitive capabilities of robots like PAL TIAGo and PAL ARI. He obtained his PhD in Cognitive Robotics in 2012 from the CNRS/LAAS and the Technical University of Munich, and worked at Bristol Robotics Lab as Associate Professor in Social Robotics, before moving to industry. His research primarily concerns socio-cognitive human-robot interaction, child-robot interaction and human-in-the-loop machine learning for social robots.

Episode 27 – Simon WanstallClaire chatted to Simon Wanstall from the Edinburgh Centre for Robotics all about soft robotics, robotic prostheses, and taking inspiration from nature.

Simon Wanstall is a PhD student at the Edinburgh Centre for Robotics, working on advancements in soft robotic prosthetics. His research interests include soft robotics, bioinspired design and healthcare devices. Simon’s current project is to develop soft sensors so that robotic prostheses can feel the world around them. In order to develop his skills in this area, Simon is also undertaking an industrial placement with Touchlab, a robotics company specialising in sensors.

Episode 28 – Amanda ProrokClaire chatted to Amanda Prorok from the University of Cambridge all about self-driving cars, industrial robots, and multi-robot systems.

Amanda Prorok is Professor of Collective Intelligence and Robotics in the Department of Computer Science and Technology at Cambridge University, and a Fellow of Pembroke College. She is interested in finding practical methods for hard coordination problems that arise in multi-robot and multi-agent systems.

Episode 29 – Sina SarehClaire chatted to Sina Sareh from the Royal College of Art all about industrial inspection, soft robotics, and robotic grippers.

Sina Sareh is the Academic Leader in Robotics at Royal College of Art. He is currently a Reader (Associate Professor) in Robotics and Design Intelligence at RCA, and a Fellow of EPSRC, whose research develops technological solutions to problems of human safety, access and performance involved in a range of industrial operations. Dr Sareh holds a PhD from the University of Bristol, 2012, and served as an impact assessor of Sub-panel 12: Engineering in the assessment phase of the Research Excellence Framework (REF) 2021.

Episode 30 – Ana CavalcantiClaire chatted to Ana Cavalcanti from the University of York all about software development, testing and verification, and autonomous mobile robots.

Ana Cavalcanti is a Royal Academy of Engineering Chair in Emerging Technologies. She is the leader of the RoboStar centre of excellence on Software Engineering for Robotics. The RoboStar approach to model-based Software Engineering complements current practice of design and verification of robotic systems, covering simulation, testing, and proof. It is practical, supported by tools, and yet mathematically rigorous.

Bonus winter treatsWhat is your favourite fictional robot?

What is your advice for a robotics career?

What is your favourite machine or tool?

Could you be friends with a robot?

A day in the life

View Details

Elvis Nava is a fellow at ETH’ Zurich’s AI center as well as a doctoral student at the Institute of Neuroinformatics and in the Soft Robotics Lab. (Photograph: Daniel Winkler / ETH Zurich)

By Christoph Elhardt

In ETH Zurich’s Soft Robotics Lab, a white robot hand reaches for a beer can, lifts it up and moves it to a glass at the other end of the table. There, the hand carefully tilts the can to the right and pours the sparkling, gold-coloured liquid into the glass without spilling it. Cheers!

Computer scientist Elvis Nava is the person controlling the robot hand developed by ETH start-up Faive Robotics. The 26-year-old doctoral student’s own hand hovers over a surface equipped with sensors and a camera. The robot hand follows Nava’s hand movement. When he spreads his fingers, the robot does the same. And when he points at something, the robot hand follows suit.

But for Nava, this is only the beginning: “We hope that in future, the robot will be able to do something without our having to explain exactly how,” he says. He wants to teach machines to carry out written and oral commands. His goal is to make them so intelligent that they can quickly acquire new abilities, understand people and help them with different tasks.

Functions that currently require specific instructions from programmers will then be controlled by simple commands such as “pour me a beer” or “hand me the apple”. To achieve this goal, Nava received a doctoral fellowship from ETH Zurich’s AI Center in 2021: this program promotes talents that bridges different research disciplines to develop new AI applications. In addition, the Italian – who grew up in Bergamo – is doing his doctorate at Benjamin Grewe’s professorship of neuroinformatics and in Robert Katzschmann’s lab for soft robotics.

Developed by the ETH start-​up Faive Robotics, the robot hand imitates the movements of a human hand. (Video: Faive Robotics)

Combining sensory stimuliBut how do you get a machine to carry out commands? What does this combination of artificial intelligence and robotics look like? To answer these questions, it is crucial to understand the human brain.

We perceive our environment by combining different sensory stimuli. Usually, our brain effortlessly integrates images, sounds, smells, tastes and haptic stimuli into a coherent overall impression. This ability enables us to quickly adapt to new situations. We intuitively know how to apply acquired knowledge to unfamiliar tasks.

“Computers and robots often lack this ability,” Nava says. Thanks to machine learning, computer programs today may write texts, have conversations or paint pictures, and robots may move quickly and independently through difficult terrain, but the underlying learning algorithms are usually based on only one data source. They are – to use a computer science term – not multimodal.

For Nava, this is precisely what stands in the way of more intelligent robots: “Algorithms are often trained for just one set of functions, using large data sets that are available online. While this enables language processing models to use the word ‘cat’ in a grammatically correct way, they don’t know what a cat looks like. And robots can move effectively but usually lack the capacity for speech and image recognition.”

“Every couple of years, our discipline changes the way we think about what it means to be a researcher,” Elvis Nava says. (Video: ETH AI Center)

Robots have to go to preschoolThis is why Nava is developing learning algorithms for robots that teach them exactly that: to combine information from different sources. “When I tell a robot arm to ‘hand me the apple on the table,’ it has to connect the word ‘apple’ to the visual features of an apple. What’s more, it has to recognise the apple on the table and know how to grab it.”

But how does the Nava teach the robot arm to do all that? In simple terms, he sends it to a two-stage training camp. First, the robot acquires general abilities such as speech and image recognition as well as simple hand movements in a kind of preschool.

Open-source models that have been trained using giant text, image and video data sets are already available for these abilities. Researchers feed, say, an image recognition algorithm with thousands of images labelled ‘dog’ or ‘cat.’ Then, the algorithm learns independently what features – in this case pixel structures – constitute an image of a cat or a dog.

A new learning algorithm for robotsNava’s job is to combine the best available models into a learning algorithm, which has to translate different data, images, texts or spatial information into a uniform command language for the robot arm. “In the model, the same vector represents both the word ‘beer’ and images labelled ‘beer’,” Nava says. That way, the robot knows what to reach for when it receives the command “pour me a beer”.

Researchers who deal with artificial intelligence on a deeper level have known for a while that integrating different data sources and models holds a lot of promise. However, the corresponding models have only recently become available and publicly accessible. What’s more, there is now enough computing power to get them up and running in tandem as well.

When Nava talks about these things, they sound simple and intuitive. But that’s deceptive: “You have to know the newest models really well, but that’s not enough; sometimes getting them up and running in tandem is an art rather than a science,” he says. It’s tricky problems like these that especially interest Nava. He can work on them for hours, continuously trying out new solutions.

Nava spends the majority of his time coding. (Photograph: Elvis Nava)

Nava evaluates his learning algorithm. The results of the experiment in a nutshell. (Photograph: Elvis Nava)

Special training: Imitating humansOnce the robot arm has completed preschool and has learnt to understand speech, recognise images and carry out simple movements, Nava sends it to special training. There, the machine learns to, say, imitate the movements of a human hand when pouring a glass of beer. “As this involves very specific sequences of movements, existing models no longer suffice,” Nava says.

Instead, he shows his learning algorithm a video of a hand pouring a glass of beer. Based on just a few examples, the robot then tries to imitate these movements, drawing on what it has learnt in preschool. Without prior knowledge, it simply wouldn’t be able to imitate such a complex sequence of movements.

“If the robot manages to pour the beer without spilling, we tell it ‘well done’ and it memorises the sequence of movements,” Nava says. This method is known as reinforcement learning in technical jargon.

Elvis Nava teaches robots to carry out oral commands such as “pour me a beer”. (Photograph: Daniel Winkler / ETH Zürich)

Foundations for robotic helpersWith this two-stage learning strategy, Nava hopes to get a little closer to realising the dream of creating an intelligent machine. How far it will take him, he does not yet know. “It’s unclear whether this approach will enable robots to carry out tasks we haven’t shown them before.”

It is much more probable that we will see robotic helpers that carry out oral commands and fulfil tasks they are already familiar with or that closely resemble them. Nava avoids making predictions as to how long it will take before these applications can be used in areas such as the care sector or construction.

Developments in the field of artificial intelligence are too fast and unpredictable. In fact, Nava would be quite happy if the robot would just hand him the beer he will politely request after his dissertation defence.

View Details

Image generated by OpenAI’s DALL-E 2 with prompt “a robot surrounded by humans, Santa Claus and a Christmas tree at Christmas, digital art”.

Happy holidays everyone! And many thanks to all those that sent us their holiday videos. Here are some robot videos of this year to get you into the spirit of the season. We wish you the very best for these holidays and the year 2023 :)

And here are some very special season greetings from robots!

Extra: How robots prepare an Amazon warehouse for Christmas

Did we miss your video? You can send it to daniel.carrillozapata@robohub.org and we’ll include it in this list.

View Details

Many people associate prosthetic limbs with nude-colored imitations of human limbs. Something built to blend into a society where people have all of their limbs while serving functional use cases. On the other end of the spectrum are the highly optimized prosthetics used by Athletes, built for speed, low weight, and appearing nothing like a human limb.

As a child under 12 years old, neither of these categories of prosthetics particularly speaks to you. Open Bionics, founded by Joel Gibbard and Samantha Payne, was started to create a third category of prosthetics. One that targets the fun, imaginative side of children, while still providing the daily functional requirements.

Through partnerships with Disney and Lucasfilms, Open Bionics has built an array of imagination-capturing prosthetic limbs that are straight-up cool.

Joel Gibbard dives into why they founded Open Bionics, and why you should invest in their company as they are getting ready to let the general public invest in them for the first time.

Joel GibbardJoel Gibbard lives in Bristol, UK and graduated with a first-class honors degree in Robotics from the University of Plymouth, UK.

He co-founded Open Bionics alongside Samantha Payne with the goal of bringing advanced, accessible bionic arms to the market. Open Bionics offers the Hero Arm, which is available in the UK, USA, France, Australia, and New Zealand. Open Bionics is revolutionizing the prosthetics industry through its line of inspiration-capturing products.

Links* Open Bionics * Pre-register to invest * Download mp3 * Subscribe to Robohub using iTunes, RSS, or Spotify * Support us on Patreon

View Details

On 9 December, CLAIRE and euRobotics jointly hosted an All Questions Answered (AQuA) event. This one hour session focussed on humanoid robotics, and participants could ask questions regarding the current and future state of AI, robotics and human augmentation in Europe.

The questions were fielded by an expert panel, comprising:

• Rainer Bischoff, euRobotics
• Wolfram Burgard, Professor of Robotics and AI, University of Technology Nuremberg
• Francesco Ferro, CEO, PAL Robotics
• Holger Hoos, Chair of the Board of Directors, CLAIRE

The session was recorded and you can watch in full below:

View Details

After 12 years of activity, NCCR Robotics officially ended on 30 November 2022.

We can proudly say that NCCR Robotics has had a truly transformational effect on the national robotics research landscape, creating novel synergies, strengthening key areas, and adding a unique signature that made Switzerland prominent and attractive at the international level.

In its 12 years of activity, NCCR Robotics has had a transformational effect on the national robotics research landscape.

Our highlights include:

• Achieving several breakthroughs in wearable, rescue and educational robotics
• Creating new master’s and doctoral programmes that will train generations of future robotics engineers
• Graduating more than 200 PhD students and 100 postdocs, with more than 1’000 peer-reviewed publications
• Spinning out several projects into companies, many of which have become international leaders and generated more than 400 jobs
• Improving awareness of gender balance in robotics and substantially increasing the percentage of women in robotics in Switzerland
• Kick-starting large outreach programs, such as Cybathlon, Swiss Drone Days, and Swiss Robotics Days, which will continue to increase public awareness of robotics for good

It is not the end of the story though: our partner institutions – EPFL, ETH Zurich, the University of Zurich, the University of Bern, the University of Basel, Università della Svizzera Italiana, EMPA – will continue to collaborate through the Innovation Booster Robotics, a new national program aimed at developing technology transfer activities and maintaining the network.

ResearchThe research programme of NCCR Robotics has been articulated around three Grand Challenges for future intelligent robots that can improve the quality of life: Wearable Robotics, Rescue Robotics, and Educational Robotics.

In the Wearable Robotics Grand Challenge, NCCR Robotics studied and developed a large range of novel prosthetic and orthotic robots, implantable sensors, and artificial intelligence algorithms to restore the capabilities of persons with disabilities and neurological disorders.

For example, researchers developed implantable and assistive technologies that allowed patients with completely paralyzed legs to walk again thanks to a combination of assistive robots (such as Rysen), implantable microdevices that read brain signals and stimulate spinal cord nerves, and artificial intelligence that translate neural signals into gait patterns.

They also developed prosthetic hands with soft sensors and implantable neural stimulators that enable people to feel again the haptic qualities of objects. Along this line, they also studied and developed prototypes of an extra arm and artificial intelligence that could allow humans to control the additional artificial limb in combination with their natural arms for situations that normally require more than one person.

Researchers also developed the MyoSuit textile soft exoskeletons that allow persons on wheelchairs to stand up, take a few steps, and then sit back on the wheelchair without external help.

In the Rescue Robotics Grand Challenge, researchers developed and deployed legged and flying robots with self-learning capabilities for use in disaster mitigation as well as in civil and industrial inspection.

Among the most notable results are ANYMal, a quadruped robot that won the first prize in the international DARPA challenge, by exploring underground tunnels and identifying a number of objects; K-rock, an amphibious robot inspired by salamanders and crocodiles that can swim, walk, and squat under narrow passages; a collision-resilient drone that has become the most widely used robot in the world by rescue teams, governments, and companies for inspection of confined spaces, bridges, boilers, and ship tankers, to mention a few; a whole family of foldable drones that can change shape to squeeze through narrow passage, protect nearby persons from propellers, carry cargo of various size and weight, and twist their arms to get close to surfaces and perform repair operations; an avian-inspired drone with artificial feathers that can approximate the flight agility of birds of prey.

In addition, researchers developed powerful learning algorithms that enabled legged robots to walk up mountains and grassy land by adapting their gait, and flying robots that learn to fly and avoid high-speed moving objects using bio-inspired vision systems, and even learned to race through a circuit beating world-champion humans. Researchers also proposed new methods to let inexperienced humans and rescue officers interact with, and easily control, drones as if they were an extension of their own body.

In the Educational Robotics Grand Challenge, NCCR researchers created Thymio, a mobile robot for teaching programming and elements of robotics that has been deployed in more than 80’000 units in classrooms across Switzerland, and Cellulo, a smaller modular robot that allows richer forms of interaction with pupils, as wells as a broad range of learning activities (physics, mathematics, geography, games) and training methods for teaching teachers how to integrate robots in their lectures.

Researchers also teamed with Canton Vaud on a large-scale project to introduce robotics and computer science into all primary-school classes and have already trained more than one thousand teachers.

OutreachCommunication, knowledge (and technology) transfer to society and the economyOver 12 years, NCCR Robotics researchers published approximately 500 articles in peer-review journals and 500 articles in peer-reviewed conferences and filed approximately 50 patents (one third of which have already been granted). They also developed a tech-transfer support programme to help young researchers translate research results into commercially viable products. As a result, 16 spin-offs were supported, out of which 14 have been incorporated and are still active. Some of these start-ups have become full scale-up companies with products sold all over the world, have raised more than five times capital than the total funds of the NCCR over 12 years, and generated several hundreds of new high-tech jobs in Switzerland.

Several initiatives were aimed at the public to communicate the importance of robotics for the quality of life.

• For example, during the first phase NCCR Robotics organized an annual Robotics Festival at EPFL that attracted at its peak 17’000 visitors in one day.
• In the second phase, Cybathlon was launched, a world-first Olympic-style competition for athletes with disabilities and supported by assistive devices, which was later taken over by ETH Zurich that will ensure its continuation.
• Additionally, NCCR Robotics launched the Swiss Drone Days at EPFL that combine drone exhibitions, drone races, and public presentations, and were later taken over by EPFL and most recently by University of Zurich.
• NCCR Robotics also organized the annual Swiss Robotics Day with the aim of bringing together researchers and industry representatives in a full day of high-profile technology presentations from top Swiss and international speakers, demonstrations of research prototypes and robotics products, carousel of pitch presentations by young spin-offs, and several panel discussions and networking events.

Promotion of young scientists and of academic careers of womenThe NCCR Robotics helped develop a new master’s programme and a new PhD programme in robotics at EPFL, created exchange programmes and fellowships with ETH Zurich and top international universities with a program in robotics, and issued several awards for excellence in study, research, technology transfer, and societal impact.

NCCR Robotics has also been very active in addressing equal opportunities. Activities aimed at improving gender balance included dedicated exchange and travel grants, awards for supporting career development, master study fellowships, outreach campaigns, promotional movies and surveys for continuous assessment of the effectiveness of actions. As a result, the percentage of women in the EPFL robotics masters almost doubled in four years, and the number of women postgraduate and assistant professors doubled too. Although much remains to be done in Switzerland, the initial results of these actions are promising and the awareness of the importance of equal opportunity has become pervasive throughout the NCCR Robotics community and in all its research, outreach, and educational activities.

Beyond NCCR RoboticsIn order to sustain the long-term impact of NCCR Robotics, EPFL launched a new Center of Intelligent Systems where robotics is a major research pillar and ETH Zurich created a Center for Robotics that includes large research facilities, annual summer schools, and activities to favor collaborations with industry.

Furthermore, EPFL built on NCCR Robotics’ educational technologies and further programmes to create the LEARN Center that will continue training teachers in the use of robots and digital technologies in schools. Similarly, ETH Zurich built on the research and competence developed in the Wearable Robotics Grand Challenge to create the Competence Centre for Rehabilitation Engineering and Science with the goal of restoring and maintaining independence, productivity, and quality of life for people with physical disabilities and contribute towards an inclusive society.

As the project approaches its conclusion, NCCR Robotics members applied for additional funding for the National Thematic Network Innovation Booster Robotics that was launched in 2022 and will continue supporting networking activities and technology transfer in medical and mobile robotics for the next 4 years. Finally, a Swiss Robotics Association comprising stakeholders from academia and industry will be created in order to manage the Innovation Booster programme and offer a communication and collaboration platform for the transformed and enlarged robotics community that NCCR Robotics has contributed to create.

View Details

Soft robots, or those made with materials like rubber, gels and cloth, have advantages over their harder, heavier counterparts, especially when it comes to tasks that require direct human interaction. Robots that could safely and gently help people with limited mobility grocery shop, prepare meals, get dressed, or even walk would undoubtedly be life-changing.

However, soft robots currently lack the strength needed to perform these sorts of tasks. This long-standing challenge — making soft robots stronger without compromising their ability to gently interact with their environment — has limited the development of these devices.

With the relationship between strength and softness in mind, a team of Penn Engineers has devised a new electrostatically controlled clutch which enables a soft robotic hand to be able to hold 4 pounds – about the weight of a bag of apples – which is 40 times more than the hand could lift without the clutch. In addition, the ability to perform this task requiring both a soft touch and strength was accomplished with only 125 volts of electricity, a third of the voltage required for current clutches.

Their safe, low-power approach could also enable wearable soft robotic devices that would simulate the sensation of holding a physical object in augmented- and virtual-reality environments.

James Pikul, Assistant Professor in Mechanical Engineering and Applied Mechanics (MEAM), Kevin Turner, Professor and Chair of MEAM with a secondary appointment in Materials Science Engineering, and their Ph.D. students, David Levine, Gokulanand Iyer and Daelan Roosa, published a study in Science Robotics describing a new, fracture-mechanics-based model of electroadhesive clutches, a mechanical structure that can control the stiffness of soft robotic materials.

Using this new model, the team was able to realize a clutch 63 times stronger than current electroadhesive clutches. The model not only increased force capacity of a clutch used in their soft robots, it also decreased the voltage required to power the clutch, making soft robots stronger and safer.

Current soft robotic hands can hold small objects, such as an apple for example. Being soft, the robotic hand can delicately grasp objects of various shapes, understand the energy required to lift them, and become stiff or tense enough to pick an object up, a task similar to how we grasp and hold things in our own hands. An electroadhesive clutch is a thin device that enhances the change of stiffness in the materials which allows the robot to perform this task. The clutch, similar to a clutch in a car, is the mechanical connection between moving objects in the system. In the case of electroadhesive clutches, two electrodes coated with a dielectric material become attracted to each other when voltage is applied. The attraction between the electrodes creates a friction force at the interface that keeps the two plates from slipping past each other. The electrodes are attached to the flexible material of the robotic hand. By turning the clutch on with an electrical voltage, the electrodes stick to each other, and the robotic hand holds more weight than it could previously. Turning the clutch off allows the plates to slide past each other and the hand to relax, so the object can be released.

Traditional models of clutches are based on a simple assumption of Coulombic friction between two parallel plates, where friction keeps the two plates of the clutch from sliding past each other. However, this model does not capture how mechanical stress is nonuniformly distributed in the system, and therefore, does not predict clutch force capacity well. It is also not robust enough to be used to develop stronger clutches without using high voltages, expensive materials, or intensive manufacturing processes. A robotic hand with a clutch created using the friction model may be able to pick up an entire bag of apples, but will require high voltages which make it unsafe for human interaction.

“Our approach tackles the force capacity of clutches at the model level,” says Pikul. “And our model, the fracture-mechanics-based model, is unique. Instead of creating parallel plate clutches, we based our design on lap joints and examined where fractures might occur in these joints. The friction model assumes that the stress on the system is uniform, which is not realistic. In reality, stress is concentrated at various points, and our model helps us understand where those points are. The resulting clutch is both stronger and safer as it requires only a third of the voltage compared to traditional clutches.”

“The fracture mechanics framework and model in this work have been used for the design of bonded joints and structural components for decades,” says Turner. “What is new here is the application of this model to the design of electroadhesive clutches.”

The researchers’ improved clutch can now be easily integrated into existing devices.

“The fracture-mechanics-based model provides fundamental insight into the workings of an electroadhesive clutch, helping us understand them more than the friction model ever could,” says Pikul. “We can already use the model to improve current clutches just by making very slight changes to material geometry and thickness, and we can continue to push the limits and improve the design of future clutches with this new understanding.”

To demonstrate the strength of their clutch, the team attached it to a pneumatic finger. Without the researchers’ clutch, the finger was able to hold the weight of one apple while inflated into a curled position; with it, the finger could hold an entire bag of them.

In another demonstration, the clutch was able to increase the strength of an elbow joint to be able to support the weight of a mannequin arm at the low energy demand of 125 volts.

Future work that the team is excited to delve into includes using this new clutch model to develop wearable augmented and virtual-reality devices.

“Traditional clutches require about 300 volts, a level that can be unsafe for human interaction,” says Levine. “We want to continue to improve our clutches, making them smaller, lighter and less energetically costly to bring these products to the real world. Eventually, these clutches could be used in wearable gloves that simulate object manipulation in a VR environment.”

“Current technologies provide feedback through vibrations, but simulating physical contact with a virtual object is limited with today’s devices,” says Pikul. “Imagine having both the visual simulation and feeling of being in another environment. VR and AR could be used in training, remote working, or just simulating touch and movement for those who lack those experiences in the real world. This technology gets us closer to those possibilities.”

Improving human-robot interactions is one of the main goals of Pikul’s lab and the direct benefits that this research presents is fuel for their own research passions.

“We haven’t seen many soft robots in our world yet, and that is, in part, due to their lack of strength, but now we have one solution to that challenge,” says Levine. “This new way to design clutches might lead to applications of soft robots that we cannot imagine right now. I want to create robots that help people, make people feel good, and enhance the human experience, and this work is getting us closer to that goal. I’m really excited to see where we go next.”

• PAPER – A mechanics-based approach to realize high–force capacity electroadhesives for robots. David J. Levine, Gokulanand M. Iyer, R. Daelan Roosa, Kevin T. Turner, and James H. Pikul. Science Robotics, 7(72), eabo2179.

View Details

That’s right! You better not run, you better not hide, you better watch out for brand new robot holiday videos on Robohub!

Drop your submissions down our chimney at daniel.carrillozapata@robohub.org and share the spirit of the season.

Here are our first two submissions of the roundup:

View Details

Shriya Srinivasan as a dancer and researcher | Snapshots taken from ‘Connecting the human body to the outside world’ video on YouTube

“The human body is just engineered so beautifully,” says Shriya Srinivasan PhD ’20, a research affiliate at MIT’s Koch Institute for Integrative Cancer Research, a junior fellow at the Society of Fellows at Harvard University, and former doctoral student in the Harvard-MIT Program in Health Sciences and Technology.

Both a biomedical engineer and a dancer, Srinivasan is dedicated to investigating the body’s movements and sensations. As a PhD student she worked in Media Lab Professor Hugh Herr’s Biomechatronics Group on a system that helps patients with amputation feel what their prostheses are feeling and send feedback from the device to the body. She has also studied the south Indian classical dance form Bharathanatyam for 22 years and co-directs the Anubhava Dance Company.

“The kind of relief and sense of fulfillment I get from the arts is very different from what I get from research and science,” she says. “I find that research often nourishes my intellectual curiosity, and the arts are helping to build that emotional and spiritual growth. But in both worlds, I’m thinking about how we create a sense of feeling, how we control emotion and your physiological response. That’s really beautiful to me.”

Video by: Jason Kimball/MIT News | 5 minutes 34 seconds.

View Details

China’s massive investment in industrial robotics has put the country in the top ranking of robot density, surpassing the United States for the first time. The number of operational industrial robots relative to the number of workers hit 322 units per 10,000 employees in the manufacturing industry. Today, China ranks in fifth place. The world´s top 5 most automated countries in manufacturing 2021 are: South Korea, Singapore, Japan, Germany and China.

World average of robot density more than doubles compared to six years ago (2015: 69 units)

“Robot density is a key indicator of automation adoption in the manufacturing industry around the world,” says Marina Bill, President of the International Federation of Robotics. “The new average of global robot density in the manufacturing industry surged to 141 robots per 10,000 employees – more than double the number six years ago. China’s rapid growth shows the power of its investment so far, but it still has much opportunity to automate.”

Robot density by regionDriven by the high volume of robot installations in recent years, Asia’s average robot density surged by 18% compound annual growth rate (CAGR) since 2016 to 156 units per 10,000 employees in 2021. The European robot density had been growing by 8% (CAGR) in the same period of time reaching 129 units. In the Americas it was 117 robots – plus 8% (CAGR).

Top countriesThe Republic of Korea hit an all-time high of 1,000 industrial robots per 10,000 employees in 2021. This is more than three times the number reached in China and makes the country number one worldwide. With its globally recognized electronics industry and a distinct automotive sector, the Korean economy profits from two large customer industries for industrial robots.

Singapore takes second place with a rate of 670 robots per 10,000 employees in 2021. Singapore’s robot density had been growing by 24% on average each year since 2016.

There is a remarkable gap to Japan (399 robots per 10,000 employees) which ranks third. Japan’s robot density had grown by 6% on average each year since 2016. Germany in fourth place (397 units) is the largest robot market in Europe.

China is by far the fastest growing robot market in the world. The country has the highest number of annual installations, and since 2016 it has each year had the largest operational stock of robots.

United StatesRobot density in the United States rose from 255 units in 2020 to 274 units in 2021. The country ranks ninth in the world, down from seventh – now head-to-head with Chinese Taipei (276 units) and behind Hong Kong (304 units) and Sweden (321 units).

Orders for World Robotics 2022 Service Robots and Industrial Robots reports can be placed online. Further downloads on the content are available here.

VideosFACTS video about ROBOT DENSITY

Video of recorded World Robotics press conference

View Details

“Learning about the social implications of the technology you’re working on is really important,” says senior Austen Roberson. Photo: Jodi Hilton

By Laura Rosado | MIT News correspondent

Austen Roberson’s favorite class at MIT is 2.S007 (Design and Manufacturing I-Autonomous Machines), in which students design, build, and program a fully autonomous robot to accomplish tasks laid out on a themed game board.

“The best thing about that class is everyone had a different idea,” says Roberson. “We all had the same game board and the same instructions given to us, but the robots that came out of people’s minds were so different.”

The game board was Mars-themed, with a model shuttle that could be lifted to score points. Roberson’s robot, nicknamed Tank Evans after a character from the movie “Surf’s Up,” employed a clever strategy to accomplish this task. Instead of spinning the gears that would raise the entire mechanism, Roberson realized a claw gripper could wrap around the outside of the shuttle and lift it manually.

“That wasn’t the intended way,” says Roberson, but his outside-of-the-box strategy ending up winning him the competition at the conclusion of the class, which was part of the New Engineering Education Transformation (NEET) program. “It was a really great class for me. I get a lot of gratification out of building something with my hands and then using my programming and problem-solving skills to make it move.”

Roberson, a senior, is majoring in aerospace engineering with a minor in computer science. As his winning robot demonstrates, he thrives at the intersection of both fields. He references the Mars Curiosity Rover as the type of project that inspires him; he even keeps a Lego model of Curiosity on his desk.

“You really have to trust that the hardware you’ve made is up to the task, but you also have to trust your software equally as much,” says Roberson, referring to the challenges of operating a rover from millions of miles away. “Is the robot going to continue to function after we’ve put it into space? Both of those things have to come together in such a perfect way to make this stuff work.”

Outside of formal classwork, Roberson has pursued multiple research opportunities at MIT that blend his academic interests. He’s worked on satellite situational awareness with the Space Systems Laboratory, tested drone flight in different environments with the Aerospace Controls Laboratory, and is currently working on zero-shot machine learning for anomaly detection in big datasets with the Mechatronics Research Laboratory.

“Whether that be space exploration or something else, all I can hope for is that I’m making an impact, and that I’m making a difference in people’s lives,” says Roberson. Photo: Jodi Hilton

Even while tackling these challenging technical problems head-on, Roberson is also actively thinking about the social impact of his work. He takes classes in the Program on Science, Technology, and Society, which has taught him not only how societal change throughout history has been driven by technological advancements, but also how to be a thoughtful engineer in his own career.

“Learning about the social implications of the technology you’re working on is really important,” says Roberson, acknowledging that his work in automation and machine learning needs to address these questions. “Sometimes, we get caught up in technology for technology’s sake. How can we take these same concepts and bring them to people to help in a tangible, physical way? How have we come together as a scientific community to really affect social change, and what can we do in the future to continue affecting that social change?”

Roberson is already working through what these questions mean for him personally. He’s been a member of the National Society of Black Engineers (NSBE) throughout his entire college experience, which includes serving on the executive board for two years. He’s helped to organize workshops focused on everything from interview preparation to financial literacy, as well as social events to build community among members.

“The mission of the organization is to increase the number of culturally responsible Black engineers that excel academically, succeed professionally, and positively impact the community,” says Roberson. “My goal with NSBE was to be able to provide a resource to help everybody get to where they wanted to be, to be the vehicle to really push people to be their best, and to provide the resources that people needed and wanted to advance themselves professionally.”

In fact, one of his most memorable MIT experiences is the first conference he attended as a member of NSBE.

“Being able to see all different these people from all of these different schools able to come together as a family and just talk to each other, it’s a very rewarding experience,” Roberson says. “It’s important to be able to surround yourself with people who have similar professional goals and share similar backgrounds and experiences with you. It’s definitely the proudest I’ve been of any club at MIT.”

Looking toward his own career, Roberson wants to find a way to work on fast-paced, cutting-edge technologies that move society forward in a positive way.

“Whether that be space exploration or something else, all I can hope for is that I’m making an impact, and that I’m making a difference in people’s lives,” says Roberson. “I think learning about space is learning about ourselves as well. The more you can learn about the stuff that’s out there, you can take those lessons to reflect on what’s down here as well.”

View Details

Teleoperation is one of the longest-standing application fields in robotics. While full autonomy is still work in progress, the possibility to remotely operate a robot has already opened scenarios where humans can act in risky environments without endangering their own safety, such as when defusing explosives or decommissioning nuclear waste. It also allows one to be present and act even at great distance: underwater, in space, or inside a patient miles away from the surgeon. These are all critical applications, where skilled and qualified operators control the robot after receiving specific training to learn to use the system safely.

Teleoperation for everyone?The recent pandemic has yet made even more apparent the need for immersive telepresence and remote action also for non-expert users: not only could teleoperated robots take vitals or bring drugs to infectious patients, but we could assist our elderly living far away with chores like moving heavy stuff, or cooking, for example. Also, numerous physical jobs could be executed from home.

The recent ANA-Xprize finals have shown how far teleoperation can go (see this impressive video of the winning team), but in such situations both the perceptual and control load lie entirely on the operator. This can be quite taxing on a cognitive level: both perception and action are mediated, by cameras and robotic arms respectively, reducing the user’s situation awareness and natural eye-hand coordination. While robot sensing capabilities and actuators have undergone relevant technological progress, the interface with the user still lacks intuitive solutions facilitating the operator’s job (Rea & Seo, 2022).

Human and robot joining forcesShared control has gained popularity in recent years, as an approach championing human-machine cooperation: low-level motor control is carried out by the robot, while the human is focused on high-level action planning. To achieve such a blend, the robotic system still needs a timely way to infer the operator intention, so as to consequently assist with the execution. Usually, motor intentions are inferred by tracking arm movements or motion control commands (if the robot is operated by means of a joystick), but especially during object manipulation the hand is tightly following information collected by the gaze. In the last decades, increasing evidence in eye-hand coordination studies has shown that gaze reliably anticipates the hand movement target (Hayhoe et al., 2012), providing an early cue about human intention.

Gaze and motion features to estimate intentionsIn a contribution presented at IROS 2022 last month (Belardinelli et al., 2022), we introduced an intention estimation model that relies on both gaze and motion features. We collected pick-and-place sequences in a virtual environment, where participants could operate two robotic grippers to grasp objects on a cluttered table. Motion controllers were used to track arm motions and to grasp objects by button press. Eye movements were tracked by the eye-tracker embedded in the virtual reality headset.

Gaze features were computed by defining a Gaussian distribution centered at the gaze position and taking for each object the likelihood for it to be the target of visual attention, which was given by the cumulative distribution collected by the object bounding box. For the motion features, the hand pose and velocity were used to estimate the hand’s current trajectory which was compared to an estimated optimal trajectory to each object. The normalized similarity between the two trajectories defined the likelihood of each object to be the target of the current movement.

Figure 1: Gaze features (top) and motion features (bottom) used for intention estimation. In both videos the object highlighted in green is the most likely target of visual attention and of hand movement, respectively.

These features along with the binary grasping state were used to train two Gaussian Hidden Markov Models, one on pick and one on place sequences. For 12 different intentions (picking of 6 different objects and placing at 6 different locations) the general accuracy (F1 score) was above 80%, even for occluded objects. Importantly, for both actions already 0.5 seconds before the end of the movement a prediction with over 90% accuracy was available for at least 70% of the observations. This would allow for an assisting plan to be instantiated and executed by the robot.

We also conducted an ablation study to determine the contribution of different feature combinations. While the models with gaze, motion, and grasping features performed better in the cross validation, the improvement with respect to only gaze and grasping state was minimal. Even when checking obstacles nearby at first, in fact, the gaze was already on the target before the hand trajectory became sufficiently discriminative.

We also ascertained that our models could generalize from one hand to the other (when fed the corresponding hand motion features), hence the same models could be used to concurrently estimate each hand intention. By feeding each hand prediction to a simple rule-based framework, basic bimanual intentions could also be recognized. So, for example, reaching for an object with the left hand while the right hand is going to place the same object on the left hand is considered a bimanual handover.

Figure 2: Online intention estimation: the red frame denotes the current right-hand intention prediction, the green frame the left-hand prediction. Above the scene, the bimanual intention is shown in capital letters.

Such an intention estimation model could help an operator to execute such manipulations without focusing on selecting the parameters for the exact motor execution of the pick and place, something we don’t usually do consciously in natural eye-hand coordination, since we automated such cognitive processes. For example, once a grasping intention is estimated with enough confidence, the robot could autonomously select the best grasp and grasping position and execute the grasp, relieving the operator of carefully monitoring a grasp without tactile feedback and possibly with inaccurate depth estimation.

Further, even if in our setup motion features were not decisive for early intention prediction, they might play a larger role in more complex settings and when extending the spectrum of bimanual manipulations.

Combined with suitable shared control policies and feedback visualizations, such systems could also enable untrained operators to control robotic manipulators transparently and effectively for longer times, improving the general mental workload of remote operation.

ReferencesBelardinelli, A., Kondapally, A. R., Ruiken, D., Tanneberg, D., & Watabe, T. (2022). Intention estimation from gaze and motion features for human-robot shared-control object manipulation. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.

Hayhoe, M. M., McKinney, T., Chajka, K., & Pelz, J. B. (2012). Predictive eye movements in natural vision. Experimental brain research, 217(1), 125-136.

Rea, D. J., & Seo, S. H. (2022). Still Not Solved: A Call for Renewed Focus on User-Centered Teleoperation Interfaces. Frontiers in Robotics and AI, 9.

View Details

Earlier this month, Candy Crush celebrated its decade birthday by hosting a free party in lower Manhattan. The climax culminated with a drone light display of 500 Unmanned Ariel Vehicles (UAVs) illustrating the whimsical characters of the popular mobile game over the Hudson. Rather than applauding the decision, New York … Read More

View Details

That’s right! You better not run, you better not hide, you better watch out for brand new robot holiday videos on Robohub! Drop your submissions down our chimney at daniel.carrillozapata@robohub.org and share the spirit of the season. For inspiration, here are our lists from previous years.

View Details

The Utah Bionic Leg is a motorized prosthetic for lower-limb amputees developed by University of Utah mechanical engineering associate professor Tommaso Lenzi and his students in the HGN Lab. Lenzi’s Utah Bionic Leg uses motors, processors, and advanced artificial intelligence that all work together to give amputees more power to walk, stand-up, sit-down, and ascend […]

View Details

In mammals, the touch modality develops earlier than the other senses, yet it is a less studied sensory modality than the visual and auditory counterparts. It not only allows environmental interactions, but also, serves as an effective defense mechanism. The role of touch in mobile robot navigation has not been explored in detail. However, touch […]

View Details

New data suggest most of the growth in the wage gap since 1980 comes from automation displacing less-educated workers.

View Details

Researchers make progress toward groups of robots that could build almost anything, including buildings, vehicles, and even bigger robots.

View Details

Are you looking for a gift for the women in robotics in your life? Or the up and coming women in robotics in your family? Perhaps these suggestions from the Women in Robotics community will inspire! We hope these are also good suggestions for non binary people in robotics, and I personally reckon they are […]

View Details

The fast-changing customer demands in modern society seek flexibility, innovation and a rapid response from manufacturers and organisations that, in order to respond to market needs, are creating tools and processes in order to adopt an approach that welcomes change. That approach is found to be Agile Manufacturing – and the Trinity project is the […]

View Details

By Fabio Bergamin Scientists around the world are researching how anti-cancer drugs can most efficiently reach the tumours they target. One possibility is to use modified bacteria as “ferries” to carry the drugs through the bloodstream to the tumours. Researchers at ETH Zurich have now succeeded in controlling certain bacteria so that they can effectively […]

View Details

Growing up in Rhode Island (the Ocean State), I lived very close to the water. Over the years, I have seen the effects of sea level rise and rapid erosion. Entire houses and beaches have slowly been consumed by the tide. I have witnessed first hand how climate change is rapidly changing the ocean ecosystem. […]

View Details

Did you have the chance to attend the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022) in Kyoto? Here we bring you the papers that received an award this year in case you missed them. Congratulations to all the winners and finalists! Best Paper Award on Cognitive Robotics Gesture2Vec: Clustering Gestures using […]

View Details

Episode 20 – Paul Dominick Baniqued Claire talked to Dr Paul Dominick Baniqued from The University of Manchester all about brain-computer interface technology and rehabilitation robotics. Paul Dominick Baniqued received his PhD in robotics and immersive technologies at the University of Leeds. His research tackled the integration of a brain-computer interface with virtual reality and […]

View Details

Eando Binder’s Adam Link scifi series predates Isaac Asimov’s more famous robots, posing issues in trust, control, and intellectual property. Read more about these challenges in my Science Robotics article here. And yes, there’s a John Wick they-killed-my-dog scene in there too. Snippet for the article with some expansion: In 1939, Eando Binder began a […]

View Details

Over the course of the past year Open Robotics has taken time from our day-to-day efforts to work with our colleagues in the field to consider how the technology we develop could negatively impact society as a whole. In particular we were concerned with the weaponization of mobile robots. After a lot of thoughtful discussion, […]

View Details

By Cormac Purcell (Adjunct Senior Lecturer, UNSW Sydney) and Paul Butcher (Adjunct Professor, Southern Cross University) Australian surf lifesavers are increasingly using drones to spot sharks at the beach before they get too close to swimmers. But just how reliable are they? Discerning whether that dark splodge in the water is a shark or just, […]

View Details

Dr. Tyler Reid, CTO of Xona Space Systems, discusses a new approach to global navigation satellite systems designed for self-driving cars.

View Details

This year’s Swiss Robotics Day – an annual event run by the EPFL-led National Centre of Competence in Research (NCCR) Robotics – will be held at the Beaulieu convention center in Lausanne. For the first time, this annual event will take place over two days: the first day, on 4 November, will be reserved for […]

View Details

Using a new technology, researchers hope to create better control systems for prosthetic limbs.

View Details

Scientists have developed a theory that can explain how flying insects determine the gravity direction without using accelerometers. It also forms a substantial step in the creation of tiny, autonomous drones. Scientists have discovered a novel manner for flying drones and insects to estimate the gravity direction. Whereas drones typically use accelerometers to this end, […]

The post Big step towards tiny autonomous drones appeared first on RoboHouse.

View Details

Researchers create a method for magnetically programming materials to make cubes that are very picky about what they connect with, enabling more-scalable self-assembly.

View Details

Tiny internet-connected electronic devices are becoming ubiquitous. The so-called Internet of Things (IoT) allows our smart gadgets in the home and wearable technologies like our smart watches to communicate and operate together. IoT devices are increasingly used across all sorts of industries to drive interconnectivity and smart automation as part of the ‘fourth industrial revolution’. […]

View Details

By Suzanna Burgelman, Frontiers science writer Researchers have designed a state-of-the-art walking robot that could revolutionize large construction projects in space. They tested the feasibility of the robot for the in-space assembly of a 25m Large Aperture Space Telescope. They present their findings in Frontiers in Robotics and AI. A scaled-down prototype of the robot […]

View Details

Computer-based neural networks can learn to do tasks. A new type of material, called a mechanical neural network, applies similar ideas to a physical structure.

View Details

The sentient, murderous humanoid robot is a complete fiction, and may never become reality. But that doesn’t mean we’re safe from autonomous weapons – they are already here.

View Details

Simple microparticles can beat rhythmically together, generating an oscillating electrical current that could be used to power microrobotic devices.

View Details

The new World Robotics report shows an all-time high of 517,385 new industrial robots installed in 2021 in factories around the world. This represents a growth rate of 31% year-on-year and exceeds the pre-pandemic record of robot installation in 2018 by 22%. Today, the stock of operational robots around the globe hits a new record […]

View Details

Our Women in Robotics list turns 10 this year and we are delighted to introduce you to another amazing “50 women in robotics you need to know about” as we also celebrate Ada Lovelace Day. We have now profiled more than 300 women AND non-binary people making important contributions to robotics since the list began in 2013. This […]

View Details

Areeb, the co-founder of Glacier, gives some insight into the opaque recycling industry, and why it is unable to handle the volume of recyclable good Americans through out each year.

View Details

It was the Summer of 2013. I was all alone at home, listening to music with my headphones. All of the sudden, I could hear someone was heavily banging on the door of my apartment. I dashed. I opened the door. To my surprise, I found my neighbour all shaky. Without hesitation, she dragged me […]

View Details

Humanoid robots could be useful in all kinds of situations, but the one Elon Musk unveiled last week is far from being ready to roll out.

View Details

Cassie the robot sets 100-metre record, photo by Kegan Sims. By Steve Lundeberg Cassie the robot, invented at the Oregon State University College of Engineering and produced by OSU spinout company Agility Robotics, has established a Guinness World Record for the fastest 100 metres by a bipedal robot. Cassie clocked the historic time of 24.73 […]

View Details

A capsule that tunnels through mucus in the GI tract could be used to orally administer large protein drugs such as insulin.

View Details

There’ve been quite a few events recently focusing on Women in Robotics, Women in Manufacturing, Women in 3D Printing, in Engineering, and in Tech Leadership. One of the largest tradeshows in the US IMTS 2022 kicked off with a Women Make Manufacturing Move Reception, with Allison Grealis, President of Women in Manufacturing, Andra Keay, President of […]

View Details

The device could help scientists explore unknown regions of the ocean, track pollution, or monitor the effects of climate change.

View Details

In the future, we imagine that teams of robots will explore and develop the surface of nearby planets, moons and asteroids – taking samples, building structures, deploying instruments. Hundreds of bright research minds are busy designing such robots. We are interested in another question: how to provide the astronauts the tools to efficiently operate their […]

View Details

Robotics solutions and technologies are rapidly changing society and transforming the way we live and work – with both positive improvements and unforeseen consequences. In Robotics4EU we wish to ensure that citizens have a say when it comes to these new technologies and how they affect everyday life. Therefore, we have gathered robots which are […]

View Details

Robots that move about by themselves must be able to adapt to the dynamic and challenging conditions in a supermarket. Hernández Corbato says: “My research focuses on using artificial intelligence to make machines smarter and more reliable by teaching them symbolic knowledge. The goal is to develop robotic ‘brains’ for intelligent robots that can be […]

The post Shelf-stocking robots with independent movement appeared first on RoboHouse.

View Details

RoboCup is an international scientific initiative with the goal of advancing the state of the art of intelligent robots, AI and automation. The annual RoboCup event returned to an in-person format for 2022, taking place from 13-17 July in Bangkok. RoboCup comprises a number of leagues, with perhaps the most well-known being the soccer leagues. […]

View Details

In the previous post, we introduced task planning in robotics. This field broadly involves a set of planning techniques that are domain-independent: That is, we can design a domain which describes the world at some (typically high) level of abstraction, using some modeling language like PDDL. However, the planning algorithms themselves can be applied to […]

View Details

Deep Learning Indaba is an organization that empowers and builds communities around Artificial Intelligence and Machine Learning across Africa. Benjamin Rosman dives into how Deep Learning Indaba is impacting these communities.

View Details

Robots: humanoids, teleoperated reconfigurable robots, swarms. Recommendation: Read this classic hard sci-fi novel and expand your horizons about robots, teleoperation, and swarms. Stanislaw Lem was one of the most read science fiction authors in the world in his day, especially the 70s and 80s, though not in America because there were rarely translations from his […]

View Details

The first robots went into space over 60 years ago and robotics have become a familiar sight in many industrial settings, but it took a bit longer for robots to make their way into our homes. Robotic vacuum cleaners first started to make an appearance in homes about 20 years ago now, and according to […]

View Details

Videos of humanoid robots dancing and performing backflips in the lab notwithstanding, robots that wash your dishes and fold your laundry are still years away. A roboticist explains why.

View Details

By Redwan Dahmouche Speed and precision are two major issues in robotics and in Industry of the Future (also known as Industry 4.0). Within this framework, RoMoCo research team of AS2M department at FEMTO-ST Institute has developed MiGriBot, a miniature robot able to perform 720 pick-and-place operations of sub-millimeter objects per minute. The results of […]

View Details

Anton Grabolle / Better Images of AI / Human-AI collaboration / Licenced by CC-BY 4.0 The 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJACI-ECAI 2022) took place from 23-29 July, in Vienna. In this post, we summarise the presentation by Ana Paiva, University of Lisbon and INESC-ID. […]

View Details

Short Spotlights on three inspiring startups who exhibited at CVPR2022

View Details

Suppose we have a robot in a simple world like the one below. Let’s consider commanding our robot to perform a task such as “take the apple from the shelf and put it on the table”. I would argue we humans have pretty good intuition for how a robot could achieve this task. We could […]

View Details

Roboticists are working on real-world versions of Sox, a robot companion in Lightyear, to play therapeutic roles. Check out how Sox compares to real world robot cats (and other pet companions) in my Science Robotics article. Sox actually checks off four of the requirements for a suitable therapy companion robot given in a 2021 paper […]

View Details

The team built the robot to mimic the behaviour of rock ants that use one-to-one tuition, in which an ant that has discovered a much better new nest can teach the route there to another individual. The findings, published in the Journal of Experimental Biology, confirm that most of the important elements of teaching in […]

View Details

Prof Brendan Englot, from Stevens Institute of Technology, discusses the challenges in perception and decision making for underwater robots - especially in the field.

View Details

Closed-loop printing enhanced by machine learning. © Michal Piovarči/ISTA Using fluids for 3D printing may seem paradoxical at first glance, but not all fluids are watery. Many useful materials are more viscous, from inks to hydrogels, and thus qualify for printing. Yet their potential has been relatively unexplored due to the limited control over their […]

View Details

Once again the global robotics community rallied to provide a unique opportunity for amateurs and hobbyists to share their robotics projects alongside academics and industry professionals. Below are the recorded sessions of this year. Industrial / Automation Keynote “Matt Whelan (Ocado Technology) – The Ocado 600 Series Robot” Nye Mech Works (HAPPA) – Real Power […]

View Details

Engineers 3D print materials with networks of sensors directly incorporated.

View Details

In early 19th-century England, the Luddites rebelled against the introduction of machinery in the textile industry. The Luddites’ name originates from the mythical tale of a weaver’s apprentice called Ned Ludd who, in an act of anger against increasingly dangerous and poor working conditions, supposedly destroyed two knitting machines. Contrary to popular belief, the Luddites […]

View Details

I have loved science fiction ever since I was a kid and read all my Dad’s ancient issues of Analog Science Fiction and Fact from the 1940s. The first book I can remember reading was The Green Hills of Earth anthology by Robert Heinlein. Fast forward to the 1990s, when, as a new professor of […]

View Details

By Andrew Dunne Gardening is proven to be healthful and joyful, but as more of us discover the joys of working in the garden for the first time, some basic knowledge about plants, landscaping and soil is required to get started. What, where and when should you plant, for instance? These were some of the […]

View Details

If you’ve been subject to my posts on Twitter or LinkedIn, you may have noticed that I’ve done no writing in the last 6 months. Besides the whole… full-time job thing … this is also because at the start of the year I decided to focus on a larger coding project. At my previous job, […]

View Details

It has been a while since I’ve posted to the blog, but lately I’ve actually been working on the UBR-1 again after a somewhat long hiatus. In case you missed the earlier posts in this series: Part 1: Getting started with ROS2 Part 2: Porting robot_controllers Part 3: Porting the head camera ROS2 Humble The […]

View Details

Brady Watkins gives insights into how a large company like Softbank Robotics America approaches the robotics market.

View Details

What are you doing right now other than scrolling through this article? Do you have a cup of coffee in one hand, your phone in the other? Maybe your right hand is using your laptop mouse and your left hand is holding a snack. Have you ever thought about how often we are using both […]

View Details

In this post we bring you all the paper awards finalists and winners presented during the 2022 edition of the IEEE International Conference on Robotics and Automation (ICRA). ICRA 2022 Outstanding Paper ‘Translating Images into Maps‘, by Saha, Avishkar; Mendez Maldonado, Oscar Alejandro; Russell, Chris; Bowden, Richard. (WINNER) ‘CT-ICP: Real-time Elastic LiDAR Odometry with Loop […]

View Details

This post shows the proceedings for the 2nd annual Reddit Robotics Showcase! We’re delighted to once again offer an event for robotics enthusiasts of all ages and abilities from across the world to share their passion and their projects. This year, we are planning an event which anticipates the enthusiasm we’ve had from our community, […]

View Details

Neuroscience professor and Science Hub investigator Ted Adelson explains how simulating the sense of touch with a camera can make robots smarter.

View Details

Dr. Akira Kakugo and his team from Hokkaido University in Japan sent us a video presentation of his recent paper ‘Cooperative cargo transportation by a swarm of molecular machines’, published in Science Robotics. ‘Despite the advancements in macro-scale robots, the development of a large number of small-size robots is still challenging, which is crucial to […]

View Details

Michael Fulton describes his unique robot-to-human communication system using exaggerated body language.

View Details

By Gareth Willmer In a city in the future, a fire breaks out in a skyscraper. An alarm is triggered and a swarm of drones swoops in, surrounds the building and uses antennas to locate people inside, enabling firefighters to go straight to the stricken individuals. Just in the nick of time – no deaths […]

View Details

By Conn Hastings, science writer Honeybees use a sophisticated dance to tell their sisters about the location of nearby flowers. This phenomenon forms the inspiration for a form of robot-robot communication that does not rely on digital networks. A recent study presents a simple technique whereby robots view and interpret each other’s movements or a […]

View Details

In cooperative multi-agent reinforcement learning (MARL), due to its on-policy nature, policy gradient (PG) methods are typically believed to be less sample efficient than value decomposition (VD) methods, which are off-policy. However, so...

View Details

RoboCup 2022 kicked off yesterday, and there have already been lots of competitions within the various leagues. Many of these are livestreamed to YouTube, and the recordings are available for anyone to watch. Below are the links to the livestream (and recorded) channels for the leagues that have them. RoboCupSoccer Standard platform Small size 2D […]

View Details

Tangram Vision, FLX Solutions, Exyn Technologies, and Pollen Robotics give us a live demo at ICRA 2022.

View Details

Sales of industrial robots have reached a strong recovery: A new record of 486,800 units were shipped globally – an increase of 27% compared to the previous year. Asia/Australia saw the largest growth in demand: installations were up 33% reaching 354,500 units. The Americas increased by 27% with 49,400 units sold. Europe saw double digit […]

View Details

Inspired by fireflies, researchers create insect-scale robots that can emit light when they fly, which enables motion tracking and communication.

View Details

In this episode, Audrow Nash speaks to Dan Mantz, who is the CEO of the Robotics Education and Competition (REC) Foundation. The REC Foundation is a nonprofit that works with VEX Robotics to build interest in STEM related career opportunities. Dan speaks about the different competitions REC facilitates, why REC is a nonprofit, how REC […]

View Details

IEEE International Conference on Robotics and Automation (ICRA) has given many opportunities over the years for researchers, industries, students and the enthusiasts to network and collaborate. In a similar fashion, this year in 2022, there were great number of opportunities to involve and engage as well including networking events. A week before the conference, IEEE […]

View Details

The ROS Awards are the Oscars of the ROS world! The intention of these awards is to express recognition for contributions to the ROS community and the development of the ROS-based robot industry, and to help those contributions gain awareness. Conditio...

View Details

Brandon Gilles, the founder of Luxonis and maker of the OAK-D, describes the journey and the flexibility of the OAK-D line of products

View Details

Researchers Matthias Hofer, Michael Muehlebach and Raffaello D’Andrea have developed the one-wheel Cubli, a three-dimensional pendulum system that can balance on its pivot using a single reaction wheel. How is it possible to stabilize the two tilt angles of the system with only a single reaction wheel? The key is to design the system such […]

View Details

Ritu Raman leads the Raman Lab, where she creates adaptive biological materials for applications in medicine and machines.

View Details

An internationally leading robotics initiative that enables academia and industry to find innovative solutions to real world challenges, celebrated its success with a Hot Robotics Symposium hosted across three UK regions last week. The National Nuclear User Facility (NNUF) for Hot Robotics is a government funded initiative that supports innovation in the nuclear sector by […]

View Details

MIT scientists unveil the first open-source simulation engine capable of constructing realistic environments for deployable training and testing of autonomous vehicles.

View Details

In this episode, Audrow Nash speaks to Maria Telleria, who is a co-founder and the CTO of Canvas. Canvas makes a drywall finishing robot and is based in the Bay Area. In this interview, Maria talks about Canvas’s drywall finishing robot, how Canvas works with unions, Canvas’s business model, and about her career path. Episode […]

View Details

As part of her role as one of the IEEE ICRA 2022 Science Communication Awardees, Avie Ravendran sat down virtually with a few researchers from academia and industry attending the conference. Curious about what they had to say? Read their quotes below! “I really believe that learned methods, especially imitation and transfer learning, will enable […]

View Details

Accessbility@ICRA2022 and OhmniLabs provided three OhmniBots for the conference, allowing students, faculty and interested industry members to attend the expo and poster sessions. The deployment of telepresence robots was greatly appreciated by everyone, from people with medical conditions or visa issues preventing them from attending, to people who were able to sample a little bit […]

View Details

Have you ever seen a bridge collapse on TV? We only care about inspection and maintenance if it does not work. However, robotics are changing the inspection & maintenance landscape, and significant societal and political implications follow the magnitude of innovation. To pay due attention to these developments, the communication on innovation in robotics for […]

View Details

“The use of robots responds to the UN´s call for action,” says Milton Guerry, President of the International Federation of Robotics (IFR). “The IFR supports the United Nations´ sustainable development goals. There are fantastic new ways in which robots save resources and produce green technologies of the future.” “The transformation on the way to a […]

View Details

As one of the ICRA Science Communication Award Winner, I covered the virtual aspects of ICRA 2022. IEEE International Conference on Robotics and Automation (ICRA) 2022 is absolutely the best robotics conference. It generally covers a vast range of robotics including but not limited to perception, control, optimization, machine learning and application-robotics. In 2022, ICRA […]

View Details

One of the essential factors for widespread robotics adoption, especially in the inspection and maintenance area, is the regulatory landscape. Regulatory and legal issues should be addressed to establish effective robotics deployment legal frameworks. Common goals of boosting the widespread adoption of robotics can only be achieved by creating networks between the robotics community and […]

View Details

Dr. Sihao Sun discusses his award-winning research in the area of controlling the flight of a drone when faced with a sudden rotor failure.

View Details

Unmanned and master/slave are two terms that are offensive to many in the community. Such terms may once have been accepted by society, but not any longer, and we are pleased to see many organizations starting to use alternative terms. We call on you in 2022 to remove words with negative connotations, like the ones […]

View Details

The 39th edition of the IEEE International Conference on Robotics and Automation (ICRA) took place at the Pennsylvania Convention Center in Philadelphia (USA) and online May 23-27. ICRA 2022 brought together the world’s top researchers and companies to share ideas and advances in the fields of robotics and automation. Nearly 8,000 participants from academia and […]

View Details

This year 2022 has been the first time that I have attended ICRA, and I had never even remotely expected that it was going to be such an intense and exciting experience for me. Before starting the conference, I believed that what would interest me the most would be the state-of-the-art robots that are showcased […]

View Details

Attending ICRA is a great opportunity to see many state-of-the-art (and famous?) robots in a single venue. Indeed, a quick trip to the exhibitors’ booths is enough to get introduced to the large and diverse group of commercial robots we have today. Everything ranging from drones, manipulators, humanoid robots, small rovers, and probably, among the […]

View Details

If you are quite enthusiastic about robotics like me, then by now you will probably have finished the third season of Netflix acclaimed series Love, Death and Robots. I believe that animation, like many other artistic expressions, is an effective way to communicate the ideas and concepts that revolve around contemporary robotics. If we envision […]

View Details

Every year, ICRA gathers an astonishing number of robot makers. With a quick look at the exhibitors, one can already perceive the immense creativity and inventiveness that creators put in their robots. To my eyes, creating a robot is an art: how does a robot should look like? What would it be good for? Should […]

View Details

John McConnell discusses the research presented at ICRA 2022 to reduce drift in SLAM algorithms by incorporating overhead satellite imagery.

View Details

The IEEE International Conference on Automation and Robotics, ICRA, is the itinerant flagship conference of the IEEE Robotics and Automation Society, RAS. In its 39th edition, ICRA is being held in the Pennsylvania Convention Center, in Philadelphia, PA, USA, between May 23 and 27, 2022. ICRA started just after the birth of the IEEE Robotics […]

View Details

In this episode, Audrow Nash speaks to Dan O’Mara, who is the founder and COO of Circuit Launch and Mechlabs. Circuit Launch is a space for hardware entrepreneurs to work in Oakland, California, and Mechlabs is a project-based course to learn robotics. This interview is mostly about Mechlabs, but talks about the origins of Circuit […]

View Details

The IEEE International Conference on Robotics and Automation (ICRA), taking place simultaneously at the Pennsylvania Convention Center in Philadelphia and virtually, has just kicked off. ICRA 2022 brings together the world’s top researchers and most important companies to share ideas and advances in the fields of robotics and automation. This is the first time the […]

View Details

Sebastian Scherer from CMU's Airlab gives us a behind-the-scenes demo at ICRA of their Autonomous Flight Control AI. Their approach aims to cooperate with human pilots and act the way they would.

View Details

Yu Sun, previous chair of the Robotics Grasping and Manipulation Competition, speaks on the value that this competition brought to the robotics community.

View Details

Bill Smart, one fo the early ICRA Competition Chairs, dives into the high-level decisions involved with creating a meaningful competition.

View Details

By Florian Meyer How can a blood clot be removed from the brain without any major surgical intervention? How can a drug be delivered precisely into a diseased organ that is difficult to reach? Those are just two examples of the countless innovations envisioned by the researchers in the field of medical microrobotics. Tiny robots […]

View Details

About 5 years ago we proposed that all robots should be fitted with the robot equivalent of an aircraft Flight Data Recorder to continuously record sensor and relevant internal status data. We call this an ethical black box (EBB). We argued that an eth...

View Details

We can now offer you a telepresence robot tour of the ICRA 2022 expo hall, competitions and poster sessions, thanks to generous support from our friends at OhmniLabs. OhmniLabs build human-centric robots that elevate quality of life for billions of people worldwide, and they build all the robots right here in Silicon Valley using advanced […]

View Details

Rubber Duckies are the passengers at the Duckietown autonomous driving competition.

View Details

Deep reinforcement learning (DRL) is transitioning from a research field focused on game playing to a technology with real-world applications. Notable examples include DeepMind’s work on controlling a nuclear reactor or on improving Yout...

View Details

‘Smart socks’ that track rising distress in the wearer could improve the wellbeing of millions of people with dementia, non-verbal autism and other conditions that affect communication.

View Details

As the next edition of the Swiss Robotics Day is in preparation in Lausanne, let’s revisit the November 2021 edition, where the vitality and richness of Switzerland’s robotics scene was on full display at StageOne Event and Convention Hall in Zurich. It was the first edition of NCCR Robotics’s flagship event after the pandemic, and […]

View Details

In this episode, Audrow Nash speaks to Afreez Gan, who is the founder and CEO of MangDang; MangDang is a Chinese startup that makes Minipupper, an open source robot dog that uses the Robot Operating System (ROS). Minipupper was inspired by and built with lessons learned from the Stanford Pupper, an open source robot dog. […]

View Details

Future generations of robots will work very differently from those that assemble entire vehicles or solder electronics onto circuit boards at lightning speed on factory floors today. They will leave the factory halls and start working with people, handing them a tool at the right moment or assisting them in assembling heavy components. They will […]

View Details

The Inspection and Maintenance (I&M) Industry represents a large economic activity spanning across multiple sectors such as energy, oil & gas, water supply, transport, civil engineering, and infrastructure. RIMA project aims at bringing together Digital Innovation Hubs and Facilitators operating under a common network that allow them to join forces and competences in promoting I&M […]

View Details

I believe that one of the best ways to get the training you need for a job market in robotics is to attend tutorials at conferences like ICRA. Unlike workshops where you might listen to some work-in-progress, other workshop paper presentations and panel discussions, tutorials are exactly what they sound like. They aim to give […]

View Details

When it comes to the future of intelligent robots, the first question people ask is often: how many jobs will they make disappear? Whatever the answer, the second question is likely to be: how can I make sure that my job is not among them? In a study just published in Science Robotics, a team […]

View Details

Researchers have developed a technique that enables a robot to learn a new pick-and-place task with only a handful of human demonstrations.

View Details

Dear robotics graduate students and newcomers to robotics, If you are what I imagine delving into robotics to be like today, the majority of your time is spent as follows: navigating Slack channels while tuning into some online lectures, trying to figure out whether you should be reading more papers, coding more, or if you […]

View Details

The robotics industry is changing. The days of industrial robot arms working behind enclosures, performing pre-programmed identical tasks are coming to an end. Robots that can interact with each other and other equipment are becoming standard and robots are expanding to more aspects of our lives. My name is Brett Pipitone, and I am the […]

View Details

By Matthew Goisman/SEAS Communications Nearly one million people in the United States live with Parkinson’s disease. The degenerative condition affects the neurons in the brain that produce the neurotransmitter dopamine, which can impact motor function in multiple ways, including muscle tremors, limb rigidity and difficulty walking. There is currently no cure for Parkinson’s disease, and […]

View Details

Dr. Carolyn Matl, Research Scientist at Toyota Research Institute, explains why Interactive Perception and soft tactile sensors are critical for manipulating challenging objects such as liquids, grains, and dough.

View Details

Number of awards: 2 Amount: travel and accommodation (up to 2k) + free registration Application deadline: 22 April 2022 We are offering up to 2 Science Communication Awards to motivated roboticists keen on helping us cover ICRA. Your coverage could include videos, interviews, podcasts, blogs, social media, or art. Please apply by sending the following […]

View Details

MIT engineers Edward Adelson and Sandra Liu duo develop a robotic gripper with rich sensory capabilities.

View Details

By Sonia Roberts, with additional editing by Dharini Dutia Diligent Robotics, founded by Andrea Thomaz and Vivian Chu, develops socially intelligent automation solutions for hospitals. Moxi, their flagship robot, delivers items like medications and wound dressings between departments to save the clinical staff’s time. Diligent has just closed their Series B funding round with $30 […]

View Details

In this episode, Audrow Nash speaks to Dayo McIntosh, who is the founder of Yateou; Yateou is an early-stage startup that makes wellness products and has a customer facing robot, ADE, that personalizes customer orders. Dayo speaks about what motivated her to start Yateou, how she uses the robot arm, ADE, and about her plans […]

View Details

MIT researchers design a robot that has a trick or two up its sleeve.

View Details

Before droves of people descend on a convention center for a trade show or conference, the hall must be carefully divided up to accommodate corporate show booths, walkways for attendees, spaces for administrators/security and much more. The process of defining the layout and marking it up for construction crews is often done with humans laboriously […]

View Details

Joe Speed, VP of Product at ApexAI, dives into the current multi-year development process of bringing a car to market, and how ApexAI will transform this process into the shorter development time we see with modern technology. This technology was showcased at the Indy Autonomous Challenge where million-dollar autonomous cars raced each other on a track.

View Details

Robots are becoming a more and more important part of our home and work lives and as we come to rely on them, trust is of paramount importance. Successful teams are founded on trust, and the same is true for human-robot teams. But what does it mean to trust a robot? I’ll be chatting to […]

View Details

A highly sensitive, 3D-printed fingertip could help robots become more dexterous and improve the performance of prosthetic hands by giving them an in-built sense of touch.

View Details

By Dan Newman To transform human mobility, exoskeletons need to interact seamlessly with their user, providing the right level of assistance at the right time to cooperate with our muscles as we move. To help achieve this, University of Michigan researchers gave users direct control to customize the behavior of an ankle exoskeleton. Not only […]

View Details

So you are considering a PhD in robotics! Before you decide to apply, here are some things to consider. What is a PhD? A PhD is a terminal degree, meaning it is the highest degree that you can earn. Almost without exception, people only earn one PhD. This degree is required for some jobs, mostly […]

View Details

March 22nd, 2012 is the day it all began. That’s the day we officially incorporated the Open Source Robotics Foundation, the origin of what we now call Open Robotics. The prospect of starting a company is both scary and exciting; but starting an open-source company in a niche as specialized as robotics, now that is […]

View Details

In this episode, Audrow Nash speaks to Helen Greiner, CEO at Tertill, which makes a small solar powered weeding robot for vegetable gardens. The conversation begins with an overview of Helen’s previous robotics experience, including at as a student at MIT, Co-founder at iRobot, Founder and CEO at CyPhyWorks, and in advising government research in […]

View Details

The AI-Guided Ultrasound Intervention Device is a lifesaving technology that helps a range of users deliver complex medical interventions at the point of injury.

View Details

This relatively general post focuses on robotics-related PhD programs in the American educational system. Much of this will not apply to universities in other countries, or to other departments in American universities. This post will take you through the overall life cycle of a PhD and is intended as a basic overview for anyone unfamiliar […]

View Details

Rob Telson dives into the Akida SOC from BrainChip, which is mimicking the five senses from the human brain on a low-powered chip.

View Details

By Alexander Badri-Sprowitz, Alborz Aghamaleki Sarvestani, Metin Sitti and Linda Behringer If a Tyrannosaurus Rex living 66 million years ago featured a similar leg structure as an ostrich running in the savanna today, then we can assume bird legs stood the test of time – a good example of evolutionary selection. Graceful, elegant, powerful – […]

View Details

By Jovita Tautkevičiūtė Robotics4EU is a 3-years-long EU-funded project which advocates for the wider adoption of AI-based robots in 4 sectors: healthcare, inspection and maintenance of infrastructure, agri-food, and agile production. Thus, one of the ways in which Robotics4EU raises awareness about non-technological aspects in robotics is delivering a series of workshops to involve the […]

View Details

Congratulations to the winners of the best paper award of the International Association for Automation and Robotics in Construction 2021. The team around Cynthia Brosque, Elena Galbally, Prof. Martin Fischer, and Prof. Oussama Khatib did excellent groundwork for construction robotics. With permission, Silicon Valley Robotics is reposting the first parts of the paper below. It […]

View Details

Robotics remained at the leading edge of technology development in 2021, yet it was one hundred years earlier in 1921 that the word robot (in its modern sense) made its public debut. Czech author Karel Čapek’s play R.U.R. (Rossum’s Universal Robots) imagined a world in which humanoids called ‘roboti’ were created in a factory. Karel’s […]

View Details

In this episode, Audrow Nash speaks to Christian Fritz, CEO and founder of Transitive Robotics. Transitive Robotics makes software for building full stack robotics applications. In this conversation, they talk about how Transitive Robotic’s software works, their business model, sandboxing for security, creating a marketplace for robotics applications, and web tools, in general. Episode Links […]

View Details

Davide Scaramuzza deep dives into Event Cameras, a fundamentally new approach to capturing visual data

View Details

On March 8, International Women’s Day (IWD) we celebrate the political, socioeconomic and cultural achievements of women and the women right’s movement towards gender equality. “Whilst the social and political rights of women are greater in some places than others, there is no country where gender equality has been achieved” says Mary Evans, professor at […]

View Details

During the pandemic, members of the reddit & discord r/robotics community rallied to organize an online showcase for members of our community. What was originally envisioned as a half-day event with one mildly interesting guest speaker turned out to be a two day event with an incredible roster of participants from across the world. You […]

View Details

Axel Krieger is the Head of the Intelligent Medical Robotic Systems and Equipment (IMERSE) Lab at Johns Hopkins University, where Justin Opfermann is pursuing his PhD degree. Together with H. Saeidi, M. Kam, S. Wei, S. Leonard , M. H. Hsieh and J. U. Kang, they recently published the paper ‘Autonomous robotic laparoscopic surgery for […]

View Details

Theories from cognitive science and psychology could help humans learn to collaborate with robots faster and more effectively, scientists find.

View Details

The images made headlines around the world in late 2018. David Mzee, who had been left paralyzed by a partial spinal cord injury suffered in a sports accident, got up from his wheelchair and began to walk with the help of a walker. This was the first proof that Courtine and Bloch’s system – which […]

View Details

This is a new series looking at the detailed design of various robots. To start with we will be looking at the design of two different robots that were used for the DARPA Subterranean Challenge. Both of these robots were designed for operating in complex subterranean environments, including Caves, Mines & Urban environments. Both of […]

View Details

Inside of the development studios of San Francisco-based Dexterity, Inc. there is a robot arm that stands as tall as a human. It is placed between a conveyor belt and several wooden pallets, all of which are typical of most warehouse packing facilities. But this is no typical warehouse facility. This is the location where […]

View Details

By Rachel Gordon | MIT CSAIL If faced with the choice of sending a swarm of full-sized, distinct robots to space, or a large crew of smaller robotic modules, you might want to enlist the latter. Modular robots, like those depicted in films such as “Big Hero 6,” hold a special type of promise for […]

View Details

In this episode, Audrow Nash speaks to Tim Chung, Program Manager in the Tactical Technology Office at the Defense Advanced Research Projects Agency (DARPA), on the DARPA Subterranean (SubT) Challenge. The SubT Challenge is a robotics challenge that aims to develop innovative technologies that would augment operations underground. In this conversation, they talk about the […]

View Details

Learn how Rajant Corporation, PBE Group, and Australian Droid + Robot – as part of a #MSHA (U.S. Department of Labor)-backed mine safety mission – achieved a historic unmanned underground mine inspection at one of the US’ largest underground room and pillar limestone operations in this comprehensive IM report. Using ten ADR Explora XL unmanned […]

View Details

Suma Reddy, CEO of Future Acres, talks about her company which is focused on creating smart farming tools to reduce labor demand and increase efficiency. Suma introduces her journey into agriculture technology and the issues in current farming practices that can benefit from robotic solutions. Future Acres’s robotic harvest companion, Carry, autonomously transports crops within […]

View Details

By Tereza Šamanová In the Czech Republic, the initial network of Digital Innovation Hubs (DIHs) in 2016 was very small – it included only two pioneer hubs raised from Horizon 2020 European projects focused on support of digital manufacturing (DIGIMAT located in Kuřim was the first one) and high-performance computing (the National Supercomputing Centre IT4Innovations […]

View Details

“Transformation for robotic automation is picking up speed across traditional and new industries,” says Milton Guerry, President of the International Federation of Robotics. “More and more companies are realizing the numerous advantages robotics provides for their businesses.” 1 – Robots adopted by new industries Segments that are relatively new to automation are rapidly adopting robots. […]

View Details

By Gareth Willmer ‘It’s the scale of it – it’s a global problem. You can guarantee that any beach you walk on, you’ll find pieces of plastic,’ said James Comerford, a senior researcher in materials and nanotechnology at SINTEF, an independent research organisation in Oslo, Norway. Plastics are estimated to comprise 85% of marine litter, […]

View Details

Valentine’s Day is approaching… Do want to sneak in a robot movie to watch on date night? Do you wonder about whether robots and love is possible? Here are five recommendations for sci-fi movies with a discussion of the related real-world robotics science. And remember to check out Learn AI and Human-Robot Interaction from Asimov’s […]

View Details

By Anastasiia Nestrogaeva (Junior Consultant at the International Projects Team, Civitta) Robotics4EU is a 3-years-long EU-funded project which advocates for a wider adoption for AI-based robots in 4 sectors: healthcare, inspection and maintenance of infrastructure, agri-food, and agile production. Thus, Robotics4EU raises awareness about non-technological aspects in robotics through delivering a series of workshops to […]

View Details

In this episode, Audrow Nash speaks to Steve Macenski, who is the Open Source Robotics Engineering Lead at Samsung Research America. Steve leads the Nav2 project, which is an open source robot navigation framework. In this conversation, they talk about the problem and challenges of robot navigation, how Nav2 works at a high level, hybrid […]

View Details

Oleg Shipitko, Chief Technical Director of Evocargo, an integrated logistics service company using autonomous vehicles speaks with Kate. Oleg talks about the need for automating operational logistics inside enclosed facilities centers and how their autonomous vehicles and other operational services can greatly improve the current way we transport goods within facilities such as ports, warehouses and […]

View Details

Created by the French company ‘Aldebaran Robotics’ in 2008, and acquired by ‘Softbank Robotics Japan’ in 2015, NAO is an autonomous and programmable humanoid robot that has been successfully applied to research and development applications for children and adults. More than 13,000 NAO robots are used in more than 70 countries around the world. Pretty […]

View Details

A new drive system for flapping wing autonomous robots has been developed by a University of Bristol team, using a new method of electromechanical zipping that does away with the need for conventional motors and gears.

View Details

The 35th conference on Neural Information Processing Systems (NeurIPS2021) featured eight invited talks. In this post, we give a flavour of the final presentation. The collective intelligence of army ants, and the robots they inspire Radhika Nagpal Radhika’s research focusses on collective intelligence, with the overarching goal being to understand how large groups of individuals, […]

View Details

The DONUts platform may look like a collection of bronze-colored, futuristic coffee cups, but everything becomes clearer as they begin to move. The group of modular robots dance in a well-choreographed symphony as magnets turn on and off allowing the modules to pull or push their neighbors. Using these simple interactions, the modular robots can […]

View Details

It’s been a hard year for women all over the world, and I’d like to thank everyone who has contributed to Women in Robotics in 2021, whether you’ve simply shared information about yourself in our community #intros channel, or organized an online event, or made yourself available as an advisor in our pilot mentoring program. […]

View Details

In this episode, Audrow Nash speaks to Matt Robinson, Program Manager for ROS-Industrial Americas at the Southwest Research Institute. ROS Industrial is a group that seeks to help industrial users, for example factories, leverage ROS and its ecosystem. In this conversation, they talk about Matt’s background, the need for the ROS-Industrial group and what problems […]

View Details

Dr. Foerster explains why incorporating uncertainty into multi-agent interactions is essential to creating robust algorithms that can operate not only in games but in real-world applications.

View Details

Through manipulation, a robotic system can physically change the state of the world. This is intertwined with intelligence, which is the ability whereby such system can detect and adapt to change. In his talk, Tamim Asfour gives an overview of the developments in manipulation for robotic systems his lab has done by learning manipulation task […]

View Details

2021 produced four new scifi books with good hard science underpinning their description of robots and three where there was less science but lots of interesting ideas about robots. Not only are these books enjoyable on their own, fiction can serve as teachable moments in robots and STEM and inspire a robot-obsessed teen to read […]

View Details

By Christoph Elhardt Steep sections on slippery ground, high steps, scree and forest trails full of roots: the path up the 1,098-​metre-high Mount Etzel at the southern end of Lake Zurich is peppered with numerous obstacles. But ANYmal, the quadrupedal robot from the Robotic Systems Lab at ETH Zurich, overcomes the 120 vertical metres effortlessly […]

View Details

By Sandrine Ceurstemont If you happened to be around the coast of Dubrovnik, Croatia in September 2021, you might have spotted two robots scouring the seafloor for debris. The robots were embarking on their inaugural mission and being tested in a real-world environment for the first time, to gauge their ability to perform certain tasks […]

View Details

Congratulations to Professor Maria Gini on winning the ACM/SIGAI Autonomous Agents Research Award for 2022! This prestigious prize recognises years of research and leadership in the field of robotics and multi-agent systems. Maria Gini is Professor of Computer Science and Engineering at the University of Minnesota, and has been at the forefront of the field […]

View Details

Sci-fi nightmares of a robot apocalypse aside, autonomous weapons are a very real threat to humanity. An expert on the weapons explains how the emerging arms race could be humanity’s last.

View Details

Did you manage to watch all the holiday robot videos of 2021? If you did but are still hungry for more, I have prepared this compilation of Science Magazine videos featuring robotics research that were released during last year. Enjoy!

View Details

In this episode, Audrow Nash speaks to Charles Brian Quinn (aka, CBQ), CEO and a Co-Founder of Greenzie. Greenzie make an autonomous driving system for commercial lawn mowers. We talk about Greenzie’s autonomous mowing system, how Greenzie has worked with manufacturers to up-fit their system into commercial mowers, how Greenzie does dog-fooding, safety and standards, […]

View Details

Location

United States

See map: Google Maps

Five years to the day after I criticized Uber for testing its self-proc...

View Details

Casinos and robots? It must be CES time! A few years ago, the only robots at CES were toys. And as the robot toy makers at Ologic can attest, having your robot featured as the leading image for CES was still no guarantee that your robot would make it into production (AMP is pictured above). […]

View Details

Here are some postcards from 2021 and wishing you all the best for 2022! Founded and Funded in 2021 According to Crunchbase, 26 robotics startups were founded and funded in 2021. Many others were founded but not funded, or funded but not founded. :) AION Prosthetics Electronics, Manufacturing, Medical Device, Robotics AION Prosthetics develops a […]

View Details

Torrey Smith, Co-Founder of Endiatx, is changing the reputation endoscopies have for being uncomfortable. At Endiatx, they are developing a pill-sized robot that you swallow, which will then livestream your digestive system for a doctor to view. Our interviewer Abate dives in. Torrey Smith Torrey Smith is the Co-Founder & CEO of Endiatx, a medical […]

View Details

Happy holidays everyone! Here are some more robot videos to get you into the holiday spirit. Have a last minute holiday robot video of your own that you’d like to share? Send your submissions to daniel.carrillozapata@robohub.org And as a late entry, here we have the video from PAL Robotics:

View Details

In this episode, Audrow Nash interviews Brett Aldrich, author of SMACC and CEO of Robosoft AI. Robosoft AI develops and maintains SMACC and SMACC2, which are event-driven, behavior state machine libraries for ROS 1 and ROS 2, respectively. Brett explains SMACC, its origins, other strategies for robot control such as behavior trees, speaks about the […]

View Details

Assistant professor of civil engineering describes her career in robotics as well as challenges and promises of human-robot interactions.

View Details

How can we make a robot learn in the real world while ensuring safety? In this work, we show how it’s possible to face this problem. The key idea to exploit domain knowledge and use the constraint definition to our advantage. Following our approach, it’s possible to implement learning robotic agents that can explore and […]

View Details

The use of industrial robots in factories around the world is accelerating at a high rate: 126 robots per 10,000 employees is the new average of global robot density in the manufacturing industries – nearly double the number five years ago (2015: 66 units). This is according to the 2021 World Robot Report.

By regions, the average robot density in Asia/Australia is 134 units, in Europe 123 units and in the Americas 111 units. The top 5 most automated countries in the world are: South Korea, Singapore, Japan, Germany, and Sweden.

“Robot density is the barometer to track the degree of automation adoption in the manufacturing industry around the world,” says Milton Guerry, President of the International Federation of Robotics.

Asia The development of robot density in China is the most dynamic worldwide: Due to the significant growth of robot installations, the density rate rose from 49 units in 2015 to 246 units in 2020. Today, China’s robot density ranks 9th globally compared to 25th just five years ago.

Asia is also the home of the country with the world´s highest robot density in the manufacturing industry: the Republic of Korea has held this position since 2010. The country’s robot density exceeds the global average seven-fold (932 units per 10,000 workers). Robot density had been increasing by 10% on average each year since 2015. With its globally recognized electronics industry and a distinct automotive industry, the Korean economy is based on the two largest areas for industrial robots.

Singapore takes second place with a rate of 605 robots per 10,000 employees in 2020. Singapore’s robot density had been growing by 27% on average each year since 2015.

Japan ranked third in the world: In 2020, 390 robots were installed per 10,000 employees in the manufacturing industry. Japan is the world´s predominant industrial robot manufacturer: The production capacity of Japanese suppliers reached 174,000 units in 2020. Today, Japan´s manufacturers deliver 45% of the global robot supply.

North America Robot density in the United States rose from 176 units in 2015 to 255 units in 2020. The country ranks seventh in the world – ahead of Chinese Taipei (248 units) and China (246 units). The modernization of domestic production facilities has boosted robot sales in the United States. The use of industrial robots also aids to achieve decarbonization targets e.g. in the cost-efficient production of solar panels and in the continued transition towards electric vehicles. Several car manufacturers have announced investments to further equip their factories for new electric drive car models or to increase capacity for battery production. These major projects will create demand for industrial robots in the next few years.

Europe Europe´s most automated country is Germany – ranking 4th worldwide with 371 units. The annual supply had a share of 33% of total robot sales in Europe 2020 – 38% of Europe’s operational stock is in Germany. The German robotics industry is recovering, mainly driven by strong overseas business rather than by the domestic or European market. Robot demand in Germany is expected to grow slowly, mainly supported by demand for low-cost robots in the general industries and outside traditional manufacturing.

France has a robot density of 194 units (ranking 16th in the world), which is well above the global average of 126 robots and relatively similar compared to other EU countries like Spain (203 units), Austria (205 units) or The Netherlands (209 units). EU members like Sweden (289 units), Denmark (246 units) or Italy (224 units), have a significantly higher degree of automation in the manufacturing segment.

As the only G7 country – the UK has a robot density below the world average of 126 units with 101 units, ranking 24th. Five years ago, the UK´s robot density was 71 units. The exodus of foreign labor after Brexit increased the demand for robots in 2020. This situation is expected to prevail in near future, the modernization of the UK manufacturing industry will also be boosted by massive tax incentives, the “super-deduction”: From April 2021 until March 2023, companies can claim 130% of capital allowances as a tax relief for plant and machinery investments.

View Details

That’s right! You better not run, you better not hide, you better watch out for brand new robot holiday videos on Robohub!

Drop your submissions down our chimney at daniel.carrillozapata@robohub.org and share the spirit of the season.

For inspiration, here are our two first videos:

View Details

MIT researchers have pioneered a new fabrication technique that enables them to produce low-voltage, power-dense, high endurance soft actuators for an aerial microrobot. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

When it comes to robots, bigger isn’t always better. Someday, a swarm of insect-sized robots might pollinate a field of crops or search for survivors amid the rubble of a collapsed building.

MIT researchers have demonstrated diminutive drones that can zip around with bug-like agility and resilience, which could eventually perform these tasks. The soft actuators that propel these microrobots are very durable, but they require much higher voltages than similarly-sized rigid actuators. The featherweight robots can’t carry the necessary power electronics that would allow them fly on their own.

Now, these researchers have pioneered a fabrication technique that enables them to build soft actuators that operate with 75 percent lower voltage than current versions while carrying 80 percent more payload. These soft actuators are like artificial muscles that rapidly flap the robot’s wings.

The artificial muscles vastly improve the robot’s payload and allow it to achieve best-in-class hovering performance. Image: Kevin Chen

This new fabrication technique produces artificial muscles with fewer defects, which dramatically extends the lifespan of the components and increases the robot’s performance and payload.

“This opens up a lot of opportunity in the future for us to transition to putting power electronics on the microrobot. People tend to think that soft robots are not as capable as rigid robots. We demonstrate that this robot, weighing less than a gram, flies for the longest time with the smallest error during a hovering flight. The take-home message is that soft robots can exceed the performance of rigid robots,” says Kevin Chen, who is the D. Reid Weedon, Jr. ’41 assistant professor in the Department of Electrical Engineering and Computer Science, the head of the Soft and Micro Robotics Laboratory in the Research Laboratory of Electronics (RLE), and the senior author of the paper.

Chen’s coauthors include Zhijian Ren and Suhan Kim, co-lead authors and EECS graduate students; Xiang Ji, a research scientist in EECS; Weikun Zhu, a chemical engineering graduate student; Farnaz Niroui, an assistant professor in EECS; and Jing Kong, a professor in EECS and principal investigator in RLE. The research has been accepted for publication in Advanced Materials and is included in the jounal’s Rising Stars series, which recognizes outstanding works from early-career researchers.

Making muscles The rectangular microrobot, which weighs less than one-fourth of a penny, has four sets of wings that are each driven by a soft actuator. These muscle-like actuators are made from layers of elastomer that are sandwiched between two very thin electrodes and then rolled into a squishy cylinder. When voltage is applied to the actuator, the electrodes squeeze the elastomer, and that mechanical strain is used to flap the wing.

The rectangular microrobot, which weighs less than one-fourth of a penny, has four sets of wings that are each driven by a soft actuator. Credits: Courtesy of the researchers

The more surface area the actuator has, the less voltage is required. So, Chen and his team build these artificial muscles by alternating between as many ultrathin layers of elastomer and electrode as they can. As elastomer layers get thinner, they become more unstable.

For the first time, the researchers were able to create an actuator with 20 layers, each of which is 10 micrometers in thickness (about the diameter of a red blood cell). But they had to reinvent parts of the fabrication process to get there.

One major roadblock came from the spin coating process. During spin coating, an elastomer is poured onto a flat surface and rapidly rotated, and the centrifugal force pulls the film outward to make it thinner.

“In this process, air comes back into the elastomer and creates a lot of microscopic air bubbles. The diameter of these air bubbles is barely 1 micrometer, so previously we just sort of ignored them. But when you get thinner and thinner layers, the effect of the air bubbles becomes stronger and stronger. That is traditionally why people haven’t been able to make these very thin layers,” Chen explains.

He and his collaborators found that if they perform a vacuuming process immediately after spin coating, while the elastomer was still wet, it removes the air bubbles. Then, they bake the elastomer to dry it.

Removing these defects increases the power output of the actuator by more than 300 percent and significantly improves its lifespan, Chen says.

The researchers also optimized the thin electrodes, which are composed of carbon nanotubes, super-strong rolls of carbon that are about 1/50,000 the diameter of human hair. Higher concentrations of carbon nanotubes increase the actuator’s power output and reduce voltage, but dense layers also contain more defects.

For instance, the carbon nanotubes have sharp ends and can pierce the elastomer, which causes the device to short out, Chen explains. After much trial and error, the researchers found the optimal concentration.

Another problem comes from the curing stage — as more layers are added, the actuator takes longer and longer to dry.

“The first time I asked my student to make a multilayer actuator, once he got to 12 layers, he had to wait two days for it to cure. That is totally not sustainable, especially if you want to scale up to more layers,” Chen says.

They found that baking each layer for a few minutes immediately after the carbon nanotubes are transferred to the elastomer cuts down the curing time as more layers are added.

Best-in-class performance After using this technique to create a 20-layer artificial muscle, they tested it against their previous six-layer version and state-of-the-art, rigid actuators.

During liftoff experiments, the 20-layer actuator, which requires less than 500 volts to operate, exerted enough power to give the robot a lift-to-weight ratio of 3.7 to 1, so it could carry items that are nearly three times its weight.

“We demonstrate that this robot, weighing less than a gram, flies for the longest time with the smallest error during a hovering flight,” says Kevin Chen. Credits: Courtesy of the researchers

They also demonstrated a 20-second hovering flight, which Chen says is the longest ever recorded by a sub-gram robot. Their hovering robot held its position more stably than any of the others. The 20-layer actuator was still working smoothly after being driven for more than 2 million cycles, far outpacing the lifespan of other actuators.

“Two years ago, we created the most power-dense actuator and it could barely fly. We started to wonder, can soft robots ever compete with rigid robots? We observed one defect after another, so we kept working and we solved one fabrication problem after another, and now the soft actuator’s performance is catching up. They are even a little bit better than the state-of-the-art rigid ones. And there are still a number of fabrication processes in material science that we don’t understand. So, I am very excited to continue to reduce actuation voltage,” he says.

Chen looks forward to collaborating with Niroui to build actuators in a clean room at MIT.nano and leverage nanofabrication techniques. Now, his team is limited to how thin they can make the layers due to dust in the air and a maximum spin coating speed. Working in a clean room eliminates this problem and would allow them to use methods, such as doctor blading, that are more precise than spin coating.

While Chen is thrilled about producing 10-micrometer actuator layers, his hope is to reduce the thickness to only 1 micrometer, which would open the door to many applications for these insect-sized robots.

This work is supported, in part, by the MIT Research Laboratory of Electronics and a Mathworks Graduate Fellowship.

• PAPER – High Lift Micro-Aerial-Robot Powered by Low Voltage and Long Endurance Dielectric Elastomer Actuators. Zhijian Ren, Suhan Kim, Xiang Ji, Weikun Zhu, Farnaz Niroui, Jing Kong and Yufeng Chen, Advanced Materials. Accepted Author Manuscript 2106757. https://doi.org/10.1002/adma.202106757

View Details

Lilly interviews Stephanie Schneider, a PhD candidate at Stanford working on unconventional space robotics. Schneider explains her work on Reachbot, a long-reach crawling and anchoring robot, which repurposes extendable booms for mobile manipulation. They discuss the challenges and exciting elements of robotic prototyping for low-gravity or otherwise unique environments.

Stephanie Schneider

Stephanie Schneider is a Ph.D. candidate in Aeronautics and Astronautics in the Autonomous Systems Laboratory at Stanford, working with Professor Marco Pavone. She received her BS in Mechanical Engineering from Cornell University in 2014. Stephanie’s research interests include real-time spacecraft motion-planning, grasping and manipulation in space, and adaptive control for autonomous robotics.

Links

• Download mp3 (26.6 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

View Details

Firstly, what is Deep Tech as opposed to Tech or technology enabled? Sometimes Deep Tech is regarded as a science based startup, sometimes it is regarded as disruptive to the status quo, sometimes it is regarded just as slow and hard, capital intensive, with a long ROI horizon. Or as something that investors aren’t ready for yet. But the amount of money going into Deep Tech investing is increasing, and the pool of Deep Tech investors is increasing. One of the key points I made in a recent GIST Tech Connect Deep Tech panel is that most investors, including the most successful Tech investors are not able to invest seriously in Deep Tech startups because they lack the technical awareness and depth of commercialization experience specific to a Deep Tech startup. GIST or the Global Innovation in Science and Technology Network is the US State Department program to encourage and support global entrepreneurship.

In fact, if you do the research into the failure rates of some high profile Deep Tech startups, it seems that certain large funds have a much higher failure rate than others, so at best, their growth pathway is not compatible with Deep Tech startups. At worst, they are simply cherry picking some Deep Tech startups for their publicity value. Startups should always do their due diligence on investors and how they treat founders, particularly founders with similar startups.

Universities play a huge role in derisking, funding and commercializing Deep Tech startups but there is still a ‘Valley of Death’ in the transfer stages. And a Deep Tech startup can come out of any university but not all universities have real commercialization experience and a supportive startup ecosystem. Silicon Valley Robotics and Circuit Launch have provided a ‘halfway house’ for a lot of Deep Tech startups by providing affordable workspace with prototyping facilities and a startup ecosystem. But the first question I always ask entrepreneurs is if they have leveraged every advantage that their university connections can provide. Universities can provide greatly discounted lab space and testing facilities, also connections to scientific experts in most any field who can be leveraged as consultants and advisors.

The SBIR program, or the American Seed Fund, which is about a $4 billion non dilutive funding from the federal government in the form of R&D dollars, contracts and grants to small businesses and startups gives you the opportunity to derisk a lot of the technology very early on. You can really do a detailed scope and scan, and then couple that with the iCorps program and you get the opportunity to do deeper dives into customer discovery, to really understand if this is something that’s just a nice to have, or is it a real must have. Although the SBIR program is American based program, a lot of the countries around the world have been creating similar ones. A good example of that is EU Horizon 2020 grants.

Grants catalyze and do a certain amount to de-risk technology, extending the runway through non-dilutive funding and by creating a technology roadmap which validates the science as significant. Corporate venture funds or strategic investors also play an important role, alongside non-dilutive grant funding. Not only can they be a check, they can be a customer, they can be an advisor and a partner in the early prototype to manufacture stages. The best strategic investors play a huge role in helping Deep Tech startups succeed, because they need the technology you are creating.

Here’s a collection of tips for Deep Tech founders gathered from the GIST TechConnect Panel on Deep Tech with Nakia Melecio from Georgia Tech, Nhi Lê from WARF, Andra Keay from SVR and The Robotics Hub, G. Nagesh Rao from US Dept of State. Also tips from Six red flags that send investors running the other way by Sara Bloomberg, San Francisco Business Times. Quotes not attributed to other investors are my thoughts or recollections from the event.

Accelerator hopping “When you start going from accelerator to accelerator looking for funding, then you’re doing it wrong. Accelerators only fund you to participate in their program. Their program and mentors are the real value.” Nhi Lê, WARF Accelerator

You also dilute your equity and become uninvestable.

Taking the first check, giving away too much equity in early rounds Always negotiate terms. But don’t focus solely on the financials and at the risk of throwing away the less obvious value that a good investor can bring to you.

“Deep Tech startups may take longer to get to revenue than a traditional tech startup, so you need to think about grant funding as a source of revenue, and any contracts that help you develop part of your technology.” Nhi Lê, WARF Accelerator

Not budgeting for IP defense “Companies often say that they’re investable because they have a patent, but they haven’t budgeted anything to defend it. Your IP is only as good as your ability to defend it. Universities play a great role in protecting and defending IP that they’ve licensed.” Nhi Lê, WARF Accelerator

Not having a plan for the whole journey “When you go into your first funding meeting, you must be thinking about the long term journey, all the way to exit. It’s never going to be just one check, you’re growing a company.” G. Nagesh Rao, US Dept of Commerce

Not doing diligence on investors or accelerators “Deep tech, especially at the leading edge, is usually expensive, so it’s critical to find the right path to commercialization at scale. Good investors speed up the process and lower your burn rate.” Michael Harries, The Robotics Hub

Have your potential investors brought similar startups to market? Having that experience can make the commercialization process much faster, and it’s critical to manage your resources effectively. Constant fundraising takes founders away from product development. Also, do your investors have patient capital? Or are they needing a rapid return on investment for their current fund? Don’t assume that a well known investor or accelerator guarantees you success, or even finding a good fit with their process.

Ignorance of basic financials Overreaching on inventory, being unable to meet debts in a timely fashion, structuring the company poorly, all these things are cited by founders who’ve struggled.

Customer discovery never stops “Focus on the customer and fall in love with the customer’s problem and you’ll never go wrong.” Nakia Melecio, Georgia Tech

Do it from the start, and never stop going to market. You can’t just outsource your business development to people with better sales skills, not until you know that pain points you’re solving for your customers and you can write the scripts for them.

Not doing the research, or using vanity metrics instead of strategy “If a founder is estimating their market in the trillions of dollars they have either not done the research or they are just delusional.” Swati Chaturvedi, Co-Founder of PropelX

“Founders who are focused only on vanity metrics (growth rate and valuation) and not attuned to developing sound business models are a red flag.” Anurag Chandra, Fort Ross Ventures

Trying to skip steps “Another red flag is trying to FOMO you into moving quickly. Not only is it bad for arriving at a sound investment decision, it’s an indication of how they do business with customers and partners (ie. not invested in building long term relationships). Anurag Chandra, Fort Ross Ventures

Misrepresentation or withholding data “Investors can tell when you are avoiding details like actual product or customer development status and it may mean you are misrepresenting your business.” Caroline Winnett, Executive Director of Berkeley SkyDeck

Cofounder issues, not having a clear leader or not being open to feedback “There needs to be agreement on who is acting as CEO, and everyone needs to be aligned on that. Another red flag is not being open to advice from experts.” Caroline Winnett, Executive Director of Berkeley SkyDeck

Being disorganized “Founders should be responsive to requests for more information. It shows if they are organized and in the mindset to do a deal versus spin cycles.” Shruti Gandhi of Array Ventures

View Details

In this episode, Audrow Nash speaks to Youssef Benmokhtar, CEO of GelSight, a Boston-based company that makes high resolution tactile sensors for several industries. They talk about how GelSight’s tactile sensors work, GelSight’s new collaboration with Meta AI (formerly, Facebook AI) to manufacture a low cost touch sensor called DIGIT, on the digitization of touch, touch sensing in robotics, how GelSight is investing in community and open source software, and Youssef’s professional path in several industries.

Episode Links

• Download the episode
• Youssef Benmokhtar’s LinkedIn
• GelSight’s Website
• DIGIT’s open source page
• PyTouch library

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• Podcast RSS feed
• Full episodes
• Clips

View Details

Congratulations to Huy Ha and Shuran Song who have won the CoRL 2021 best system paper award!

Their work, FlingBot: the unreasonable effectiveness of dynamic manipulations for cloth unfolding, was highly praised by the judging committee. “To me, this paper constitutes the most impressive account of both simulated and real-world cloth manipulation to date.”, commented one of the reviewers.

Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper? In my most recent publication with my advisor, Professor Shuran Song, we studied the task of cloth unfolding. The goal of the task is to manipulate a cloth from a crumpled initial state to an unfolded state, which is equivalent to maximizing the coverage of the cloth on the workspace.

Could you tell us about the implications of your research and why it is an interesting area for study? Historically, most robotic manipulation research topics, such as grasp planning, are concerned with rigid objects, which have only 6 degrees of freedom since their geometry does not change. This allows one to apply the typical state estimation – task & motion planning pipeline in robotics. In contrast, deformable objects could bend and stretch in arbitrary directions, leading to infinite degrees of freedom. It’s unclear what the state of the cloth should even be. In addition, deformable objects such as clothes could experience severe self occlusion – given a crumpled piece of cloth, it’s difficult to identify whether it’s a shirt, jacket, or pair of pants. Therefore, cloth unfolding is a typical first step of cloth manipulation pipelines, since it reveals key features of the cloth for downstream perception and manipulation.

Despite the abundance of sophisticated methods for cloth unfolding over the years, they typically only address the easy case (where the cloth already starts off mostly unfolded) or take upwards of a hundred steps for challenging cases. These prior works all use single arm quasi-static actions, such as pick and place, which is slow and limited by the physical reach range of the system.

Could you explain your methodology? In our daily lives, humans typically use both hands to manipulate cloths, and with as little as a single high velocity fling or two, we can unfold an initially crumpled cloth. Based on this observation, our key idea is simple: Use dual arm dynamic actions for cloth unfolding.

FlingBot is a self-supervised framework for cloth unfolding which uses a pick, stretch, and fling primitive for a dual-arm setup from visual observations. There are three key components to our approach. First is the decision to use a high velocity dynamic action. By relying on cloths’ mass combined with a high-velocity throw to do most of its work, a dynamic flinging policy can unfold cloths much more efficiently than a quasi-static policy. Second is a dual-arm grasp parameterization which makes satisfying collision safety constraints easy. By treating a dual-arm grasp not as two points but as a line with a rotation and length, we can directly constrain the rotation and length of the line to ensure arms do not cross over each other and do not try to grasp too close to each other. Third is our choice of using Spatial Action Maps, which learns translational, rotational, and scale equivariant value maps, and allows for sample efficient learning.

What were your main findings? We found that dynamic actions have three desirable properties over quasi-static actions for the task of cloth unfolding. First, they are efficient – FlingBot achieves over 80% coverage within 3 actions on novel cloths. Second, they are generalizable – trained on only square cloths, FlingBot also generalizes to T-shirts. Third, they expand the system’s effective reach range – even when FlingBot can’t fully lift or stretch a cloth larger than the system’s physical reach range, it’s able to use high velocity flings to unfold the cloth.

After training and evaluating our model in simulation, we deployed and finetuned our model on a real world dual-arm system, which achieves above 80% coverage for all cloth categories. Meanwhile, the quasi-static pick & place baseline was only able to achieve around 40% coverage.

What further work are you planning in this area? Although we motivated cloth unfolding as a precursor for downstream modules such as cloth state estimation, unfolding could also benefit from state estimation. For instance, if the system is confident it has identified the shoulders of the shirt in its state estimation, the unfolding policy could directly grasp the shoulders and unfold the shirt in one step. Based on this observation, we are currently working on a cloth unfolding and state estimation approach which can learn in a self-supervised manner in the real world.

About the authors

| | Huy Ha is a Ph.D. student in Computer Science at Columbia University. He is advised by Professor Shuran Song and is a member of the Columbia Artificial Intelligence and Robotics (CAIR) lab. |

| | Shuran Song is an assistant professor in computer science department at Columbia University, where she directs the Columbia Artificial Intelligence and Robotics (CAIR) Lab. Her research focuses on computer vision and robotics. She’s interested in developing algorithms that enable intelligent systems to learn from their interactions with the physical world, and autonomously acquire the perception and manipulation skills necessary to execute complex tasks and assist people. |

Find out more * Read the paper on arXiv. * The videos of the real-world experiments and code are available here, as is a video of the authors’ presentation at CoRL. * Read more about the winning and shortlisted papers for the CoRL awards here.

View Details

At the beginning of the new millennia, wireless capsule endoscopy was introduced as a minimally invasive method of inspecting the digestive tract. The possibility of collecting images deep inside the human body just by swallowing a “pill” revolutionized the field of gastrointestinal endoscopy and sparked a brand-new field of research in robotics: medical capsule robots. These are self-contained robots that leverage extreme miniaturization to access and operate in environments that are out of reach for larger devices. In medicine, capsule robots can enter the human body through natural orifices or small incisions, and detect and cure life-threatening diseases in a non-invasive manner. This talk provides a perspective on how this field has evolved in the last ten years. We explore what was accomplished, what has failed, and what were the lessons learned. We also discuss enabling technologies, intelligent control, possible levels of computer assistance, and highlight future challenges in this ongoing Fantastic Voyage.

Bio: Pietro Valdastri (Senior Member, IEEE) received the master’s degree (Hons.) from the University of Pisa, in 2002, and the Ph.D. degree in biomedical engineering, Scuola Superiore Sant’Anna in 2006. He is a Professor and a Chair of Robotics and Autonomous Systems with the University of Leeds. His research interests include robotic surgery, robotic endoscopy, design of magnetic mechanisms, and medical capsule robots. He is a recipient of the Wolfson Research Merit Award from the Royal Society.

View Details

MIT students and researchers from MIT Sea Grant work with local oyster farmers in advancing the aquaculture industry by seeking solutions to some of its biggest challenges. Currently, oyster bags have to be manually flipped every one to two weeks to reduce biofouling. Image: John Freidah, MIT MechE

By Michaela Jarvis | Department of Mechanical Engineering

When Michelle Kornberg was about to graduate from MIT, she wanted to use her knowledge of mechanical and ocean engineering to make the world a better place. Luckily, she found the perfect senior capstone class project: supporting sustainable seafood by helping aquaculture farmers grow oysters.

“It’s our responsibility to use our skills and opportunities to work on problems that really matter,” says Kornberg, who now works for an aquaculture company called Innovasea. “Food sustainability is incredibly important from an environmental standpoint, of course, but it also matters on a social level. The most vulnerable will be hurt worst by the climate crisis, and I think food sustainability and availability really matters on that front.”

The project undertaken by Kornberg’s capstone class, 2.017 (Design of Electromechanical Robotic Systems), came out of conversations between Michael Triantafyllou, who is MIT’s Henry L. and Grace Doherty Professor in Ocean Science and Engineering and director of MIT Sea Grant, and Dan Ward. Ward, a seasoned oyster farmer and marine biologist, owns Ward Aquafarms on Cape Cod and has worked extensively to advance the aquaculture industry by seeking solutions to some of its biggest challenges.

Speaking with Triantafyllou at MIT Sea Grant — part of a network of university-based programs established by the federal government to protect the coastal environment and economy — Ward had explained that each of his thousands of floating mesh oyster bags need to be turned over about 11 times a year. The flipping allows algae, barnacles, and other “biofouling” organisms that grow on the part of the bag beneath the water’s surface to be exposed to air and light, so they can dry and chip off. If this task is not performed, water flow to the oysters, which is necessary for their growth, is blocked.

The bags are flipped by a farmworker in a kayak, and the task is monotonous, often performed in rough water and bad weather, and ergonomically injurious. “It’s kind of awful, generally speaking,” Ward says, adding that he pays about $3,500 per year to have the bags turned over at each of his two farm sites — and struggles to find workers who want to do the job of flipping bags that can grow to a weight of 60 or 70 pounds just before the oysters are harvested.

Presented with this problem, the capstone class Kornberg was in — composed of six students in mechanical engineering, ocean engineering, and electrical engineering and computer science — brainstormed solutions. Most of the solutions, Kornberg says, involved an autonomous robot that would take over the bag-flipping. It was during that class that the original version of the “Oystamaran,” a catamaran with a flipping mechanism between its two hulls, was born.

A combination of mechanical engineering, ocean engineering, and electrical engineering and computer sciences students work together to design a robot to help with flipping oyster bags at Ward Aquafarm on Cape Cod. The “Oystamaran” robot uses a vision system to position and flip the bags. Image: Lauren Futami, MIT MechE

Ward’s involvement in the project has been important to its evolution. He says he has reviewed many projects in his work on advisory boards that propose new technologies for aquaculture. Often, they don’t correspond with the actual challenges faced by the industry.

“It was always ‘I already have this remotely operated vehicle; would it be useful to you as an oyster farmer if I strapped on some kind of sensor?’” Ward says. “They try to fit robotics into aquaculture without any industry collaboration, which leads to a robotic product that doesn’t solve any of the issues we experience out on the farm. Having the opportunity to work with MIT Sea Grant to really start from the ground up has been exciting. Their approach has been, ‘What’s the problem, and what’s the best way to solve the problem?’ We do have a real need for robotics in aquaculture, but you have to come at it from the customer-first, not the technology-first, perspective.”

Triantafyllou says that while the task the robot performs is similar to work done by robots in other industries, the “special difficulty” students faced while designing the Oystamaran was its work environment.

“You have a floating device, which must be self-propelled, and which must find these objects in an environment that is not neat,” Triantafyllou says. “It’s a combination of vision and navigation in an environment that changes, with currents, wind, and waves. Very quickly, it becomes a complicated task.”

Kornberg, who had constructed the original central flipping mechanism and the basic structure of the vessel as a staff member at MIT Sea Grant after graduating in May 2020, worked as a lab instructor for the next capstone class related to the project in spring 2021. Andrew Bennett, education administrator at MIT Sea Grant, co-taught that class, in which students designed an Oystamaran version 2.0, which was tested at Ward Aquafarms and managed to flip several rows of bags while being controlled remotely. Next steps will involve making the vessel more autonomous, so it can be launched, navigate autonomously to the oyster bags, flip them, and return to the launching point. A third capstone class related to the project will take place this spring.

The students operate the “Oystamaran” robot remotely from the boat. Image: John Freidah, MIT MechE

Bennett says an ideal project outcome would be, “We have proven the concept, and now somebody in industry says, ‘You know, there’s money to be made in oysters. I think I’ll take over.’ And then we hand it off to them.”

Meanwhile, he says an unexpected challenge arose with getting the Oystamaran to go between tightly packed rows of oyster bags in the center of an array.

“How does a robot shimmy in between things without wrecking something? It’s got to wiggle in somehow, which is a fascinating controls problem,” Bennett says, adding that the problem is a source of excitement, rather than frustration, to him. “I love a new challenge, and I really love when I find a problem that no one expected. Those are the fun ones.”

Triantafyllou calls the Oystamaran “a first for the industry,” explaining that the project has demonstrated that robots can perform extremely useful tasks in the ocean, and will serve as a model for future innovations in aquaculture.

“Just by showing the way, this may be the first of a number of robots,” he says. “It will attract talent to ocean farming, which is a great challenge, and also a benefit for society to have a reliable means of producing food from the ocean.”

View Details

By Marco Arruda

In this post you’ll learn how to program a robot to avoid obstacles using ROS2 and C++. Up to the end of the post, the Dolly robot moves autonomously in a scene with many obstacles, simulated using Gazebo 11.

You’ll learn:

• How to publish AND subscribe topics in the same ROS2 Node
• How to avoid obstacles
• How to implement your own algorithm in ROS2 and C++

1 – Setup environment – Launch simulation Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create the node In order to have our obstacle avoidance algorithm, let’s create a new executable in the file ~/ros2_ws/src/my_package/obstacle_avoidance.cpp:

```

include "geometry_msgs/msg/twist.hpp" // Twist

include "rclcpp/rclcpp.hpp" // ROS Core Libraries

include "sensor_msgs/msg/laser_scan.hpp" // Laser Scan

using std::placeholders::_1;

class ObstacleAvoidance : public rclcpp::Node { public: ObstacleAvoidance() : Node("ObstacleAvoidance") {

auto default\_qos = rclcpp::QoS(rclcpp::SystemDefaultsQoS());
subscription\_ = this->create\_subscription(
    "laser\_scan", default\_qos,
    std::bind(&ObstacleAvoidance::topic\_callback, this, \_1));
publisher\_ =
    this->create\_publisher("cmd\_vel", 10);

}

private: void topic_callback(const sensor_msgs::msg::LaserScan::SharedPtr _msg) { // 200 readings, from right to left, from -57 to 57 degress // calculate new velocity cmd float min = 10; for (int i = 0; i < 200; i++) { float current = _msg->ranges[i]; if (current < min) { min = current; } } auto message = this->calculateVelMsg(min); publisher_->publish(message); } geometry_msgs::msg::Twist calculateVelMsg(float distance) { auto msg = geometry_msgs::msg::Twist(); // logic RCLCPP_INFO(this->get_logger(), "Distance is: '%f'", distance); if (distance < 1) { // turn around msg.linear.x = 0; msg.angular.z = 0.3; } else { // go straight ahead msg.linear.x = 0.3; msg.angular.z = 0; } return msg; } rclcpp::Publisher::SharedPtr publisher_; rclcpp::Subscription::SharedPtr subscription_; };

int main(int argc, char *argv[]) { rclcpp::init(argc, argv); rclcpp::spin(std::make_shared()); rclcpp::shutdown(); return 0; }

```

In the main function we have:

• Initialize node rclcpp::init
• Keep it running rclcpp::spin

Inside the class constructor:

• Subcribe to the laser scan messages: subscription_
• Publish to the robot diff driver: publisher_

The obstacle avoidance intelligence goes inside the method calculateVelMsg. This is where decisions are made based on the laser readings. Notice that is depends purely on the minimum distance read from the message.

If you want to customize it, for example, consider only the readings in front of the robot, or even check if it is better to turn left or right, this is the place you need to work on! Remember to adjust the parameters, because the way it is, only the minimum value comes to this method.

3 – Compile the node This executable depends on both geometry_msgs and sensor_msgs, that we have added in the two previous posts of this series. Make sure you have them at the beginning of the ~/ros2_ws/src/my_package/CMakeLists.txt file:

```

find dependencies

find_package(ament_cmake REQUIRED) find_package(rclcpp REQUIRED) find_package(geometry_msgs REQUIRED) find_package(sensor_msgs REQUIRED)

```

And finally, add the executable and install it:

```

obstacle avoidance

add_executable(obstacle_avoidance src/obstacle_avoidance.cpp) ament_target_dependencies(obstacle_avoidance rclcpp sensor_msgs geometry_msgs)

...

install(TARGETS reading_laser moving_robot obstacle_avoidance DESTINATION lib/${PROJECT_NAME}/ )

```

Compile the package:

colcon build --symlink-install --packages-select my_package

4 – Run the node In order to run, use the following command:

ros2 run my_package obstacle_avoidance

It will not work for this robot! Why is that? We are subscribing and publishing to generic topics: cmd_vel and laser_scan.

We need a launch file to remap these topics, let’s create one at ~/ros2_ws/src/my_package/launch/obstacle_avoidance.launch.py:

``` from launch import LaunchDescription from launch_ros.actions import Node

def generate_launch_description():

obstacle\_avoidance = Node(
    package='my\_package',
    executable='obstacle\_avoidance',
    output='screen',
    remappings=[
        ('laser\_scan', '/dolly/laser\_scan'),
        ('cmd\_vel', '/dolly/cmd\_vel'),
    ]
)

return LaunchDescription([obstacle\_avoidance])

```

Recompile the package, source the workspace once more and launch it:

`colcon build --symlink-install --packages-select my_package

source ~/ros2_ws/install/setup.bash

ros2 launch my_package obstacle_avoidance.launch.py`

Related courses & extra links: * ROS2 Basics for C++ * Copy my rosject here * Dolly robot – Official repo

The post Exploring ROS2 with a wheeled robot – #4 – Obstacle Avoidance appeared first on The Construct.

View Details

AI-designed (C-shaped) organisms push loose stem cells (white) into piles as they move through their environment. Credit: Douglas Blackiston and Sam Kriegman

By Joshua Brown, University of Vermont Communications

To persist, life must reproduce. Over billions of years, organisms have evolved many ways of replicating, from budding plants to sexual animals to invading viruses.

Now scientists at the University of Vermont, Tufts University, and the Wyss Institute for Biologically Inspired Engineering at Harvard University have discovered an entirely new form of biological reproduction—and applied their discovery to create the first-ever, self-replicating living robots.

The same team that built the first living robots (“Xenobots,” assembled from frog cells—reported in 2020) has discovered that these computer-designed and hand-assembled organisms can swim out into their tiny dish, find single cells, gather hundreds of them together, and assemble “baby” Xenobots inside their Pac-Man-shaped “mouth”—that, a few days later, become new Xenobots that look and move just like themselves.

And then these new Xenobots can go out, find cells, and build copies of themselves. Again and again.

“With the right design—they will spontaneously self-replicate,” says Joshua Bongard, Ph.D., a computer scientist and robotics expert at the University of Vermont who co-led the new research.

The results of the new research were published in the Proceedings of the National Academy of Sciences.

Into the Unknown In a Xenopus laevis frog, these embryonic cells would develop into skin. “They would be sitting on the outside of a tadpole, keeping out pathogens and redistributing mucus,” says Michael Levin, Ph.D., a professor of biology and director of the Allen Discovery Center at Tufts University and co-leader of the new research. “But we’re putting them into a novel context. We’re giving them a chance to reimagine their multicellularity.” Levin is also an Associate Faculty member at the Wyss Institute.

As Pac-man-shaped Xenobot “parents” move around their environment, they collect loose stem cells in their “mouths” that, over time, aggregate to create “offspring” Xenobots that develop to look just like their creators. Credit: Doug Blackiston and Sam Kriegman

And what they imagine is something far different than skin. “People have thought for quite a long time that we’ve worked out all the ways that life can reproduce or replicate. But this is something that’s never been observed before,” says co-author Douglas Blackiston, Ph.D., the senior scientist at Tufts University and the Wyss Institute who assembled the Xenobot “parents” and developed the biological portion of the new study.

“This is profound,” says Levin. “These cells have the genome of a frog, but, freed from becoming tadpoles, they use their collective intelligence, a plasticity, to do something astounding.” In earlier experiments, the scientists were amazed that Xenobots could be designed to achieve simple tasks. Now they are stunned that these biological objects—a computer-designed collection of cells—will spontaneously replicate. “We have the full, unaltered frog genome,” says Levin, “but it gave no hint that these cells can work together on this new task,” of gathering and then compressing separated cells into working self-copies.

“These are frog cells replicating in a way that is very different from how frogs do it. No animal or plant known to science replicates in this way,” says Sam Kriegman, Ph.D., the lead author on the new study, who completed his Ph.D. in Bongard’s lab at UVM and is now a post-doctoral researcher at Tuft’s Allen Center and Harvard University’s Wyss Institute for Biologically Inspired Engineering.

On its own, the Xenobot parent, made of some 3,000 cells, forms a sphere. “These can make children but then the system normally dies out after that. It’s very hard, actually, to get the system to keep reproducing,” says Kriegman. But with an artificial intelligence program working on the Deep Green supercomputer cluster at UVM’s Vermont Advanced Computing Core, an evolutionary algorithm was able to test billions of body shapes in simulation—triangles, squares, pyramids, starfish—to find ones that allowed the cells to be more effective at the motion-based “kinematic” replication reported in the new research.

“We asked the supercomputer at UVM to figure out how to adjust the shape of the initial parents, and the AI came up with some strange designs after months of chugging away, including one that resembled Pac-Man,” says Kriegman. “It’s very non-intuitive. It looks very simple, but it’s not something a human engineer would come up with. Why one tiny mouth? Why not five? We sent the results to Doug and he built these Pac-Man-shaped parent Xenobots. Then those parents built children, who built grandchildren, who built great-grandchildren, who built great-great-grandchildren.” In other words, the right design greatly extended the number of generations.

Kinematic replication is well-known at the level of molecules—but it has never been observed before at the scale of whole cells or organisms.

An AI-designed “parent” organism (C shape; red) beside stem cells that have been compressed into a ball (“offspring”; green). Credit: Douglas Blackiston and Sam Kriegman

“We’ve discovered that there is this previously unknown space within organisms, or living systems, and it’s a vast space,” says Bongard. “How do we then go about exploring that space? We found Xenobots that walk. We found Xenobots that swim. And now, in this study, we’ve found Xenobots that kinematically replicate. What else is out there?”

Or, as the scientists write in the Proceedings of the National Academy of Sciences study: “life harbors surprising behaviors just below the surface, waiting to be uncovered.”

Responding to Risk Some people may find this exhilarating. Others may react with concern, or even terror, to the notion of a self-replicating biotechnology. For the team of scientists, the goal is deeper understanding.

“We are working to understand this property: replication. The world and technologies are rapidly changing. It’s important, for society as a whole, that we study and understand how this works,” says Bongard. These millimeter-sized living machines, entirely contained in a laboratory, easily extinguished, and vetted by federal, state and institutional ethics experts, “are not what keep me awake at night. What presents risk is the next pandemic; accelerating ecosystem damage from pollution; intensifying threats from climate change,” says UVM’s Bongard. “This is an ideal system in which to study self-replicating systems. We have a moral imperative to understand the conditions under which we can control it, direct it, douse it, exaggerate it.”

Bongard points to the COVID epidemic and the hunt for a vaccine. “The speed at which we can produce solutions matters deeply. If we can develop technologies, learning from Xenobots, where we can quickly tell the AI: ‘We need a biological tool that does X and Y and suppresses Z,’ —that could be very beneficial. Today, that takes an exceedingly long time.” The team aims to accelerate how quickly people can go from identifying a problem to generating solutions—”like deploying living machines to pull microplastics out of waterways or build new medicines,” Bongard says.

“We need to create technological solutions that grow at the same rate as the challenges we face,” Bongard says.

And the team sees promise in the research for advancements toward regenerative medicine. “If we knew how to tell collections of cells to do what we wanted them to do, ultimately, that’s regenerative medicine—that’s the solution to traumatic injury, birth defects, cancer, and aging,” says Levin. “All of these different problems are here because we don’t know how to predict and control what groups of cells are going to build. Xenobots are a new platform for teaching us.”

The scientists behind the Xenobots participated in a live panel discussion on December 1, 2021 to discuss the latest developments in their research. Credit: Wyss Institute at Harvard University

• PUBLICATION – Kinematic self-replication in reconfigurable organisms. Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard, Proceedings of the National Academy of Sciences Dec 2021, 118 (49) e2112672118; DOI: 10.1073/pnas.2112672118

View Details

By Marco Arruda

In this post you’ll learn how to publish to a ROS2 topic using ROS2 C++. Up to the end of the video, we are moving the robot Dolly robot, simulated using Gazebo 11.

You’ll learn:

• How to create a node with ROS2 and C++
• How to public to a topic with ROS2 and C++

1 – Setup environment – Launch simulation Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create a topic publisher Create a new file to container the publisher node: moving_robot.cpp and paste the following content:

```

include

include

include

include "rclcpp/rclcpp.hpp"

include "geometry_msgs/msg/twist.hpp"

using namespace std::chrono_literals;

/ This example creates a subclass of Node and uses std::bind() to register a * member function as a callback from the timer. /

class MovingRobot : public rclcpp::Node { public: MovingRobot() : Node("moving_robot"), count_(0) { publisher_ = this->create_publisher("/dolly/cmd_vel", 10); timer_ = this->create_wall_timer( 500ms, std::bind(&MovingRobot::timer_callback, this)); }

private: void timer_callback() { auto message = geometry_msgs::msg::Twist(); message.linear.x = 0.5; message.angular.z = 0.3; RCLCPP_INFO(this->get_logger(), "Publishing: '%f.2' and %f.2", message.linear.x, message.angular.z); publisher_->publish(message); } rclcpp::TimerBase::SharedPtr timer_; rclcpp::Publisher::SharedPtr publisher_; size_t count_; };

int main(int argc, char *argv[]) { rclcpp::init(argc, argv); rclcpp::spin(std::make_shared()); rclcpp::shutdown(); return 0; }QoS (Quality of Service)

```

Similar to the subscriber it is created a class that inherits Node. A publisher_ is setup and also a callback, although this time is not a callback that receives messages, but a timer_callback called in a frequency defined by the timer_ variable. This callback is used to publish messages to the robot.

The create_publisher method needs two arguments:

• topic name
• QoS (Quality of Service) – This is the policy of data saved in the queue. You can make use of different middlewares or even use some provided by default. We are just setting up a queue of 10. By default, it keeps the last 10 messages sent to the topic.

The message published must be created using the class imported:

message = geometry_msgs::msg::Twist();

We ensure the callback methods on the subscribers side will always recognize the message. This is the way it has to be published by using the publisher method publish.

3 – Compile and run the node In order to compile we need to adjust some things in the ~/ros2_ws/src/my_package/CMakeLists.txt. So add the following to the file:

• Add the geometry_msgs dependency
• Append the executable moving_robot
• Add install instruction for moving_robot

``` find_package(geometry_msgs REQUIRED) ...

moving robot

add_executable(moving_robot src/moving_robot.cpp) ament_target_dependencies(moving_robot rclcpp geometry_msgs) ... install(TARGETS moving_robot reading_laser DESTINATION lib/${PROJECT_NAME}/ )

```

We can run the node like below:

`source ~/ros2_ws/install/setup.bash

ros2 run my_package`

Related courses & extra links: * ROS2 Basics for C++ * Copy my rosject here * Dolly robot – Official repo

*The post Exploring ROS2 using wheeled Robot – #3 – Moving the Robot

appeared first on The Construct.*

View Details

Much excellent work has been done, by many organizations, to develop ‘Roadmaps for Robotics’, in order to steer government policy, innovation investment, and the development of standards and commercialization. However, unless you took part in the roadmapping activity, it can be very hard to find these resources. Silicon Valley Robotics in partnership with the Industrial Activities Board of the IEEE Robotics and Automation Society, is compiling an up to date resource list of various robotics, AIS and AI roadmaps, national or otherwise. This initiative will allow us all to access the best robotics commercialization advice from around the world, to be able to compare and contrast various initiatives and their regional effectiveness, and to provide guidance for countries and companies without their own robotics roadmaps.

Another issue making it harder to find recent robotics roadmaps is the subsumption of robotics into the AI landscape, at least in some national directives. Or it may appear not as robotics but as ‘AIS’, standing for Autonomous Intelligent Systems, such as in the work of OCEANIS, the Open Community for Ethics in Autonomous aNd Intelligent Systems, which hosts a global standards repository. And finally there are subcategories of robotics, ie Autonomous Vehicles, or Self Driving Cars, or Drones, or Surgical Robotics, all of which may have their own roadmaps. This is not an exhaustive list, but with your help we can continue to evolve it.

• Global Roadmaps:
  • IEEE RAS 2050Robotics.org
  • IEEE Roadmaps
  • OCEANIS, Open Community for Ethics of Autonomous and Intelligent Systems
  • OCEANIS, Global Standards Repository
  • OECD.ai Policy Observatory
  • OECD.ai Stakeholder Initiatives
  • OECD.ai National AI Strategies
  • Mapping the Evolution of the Robotics Industry: A Cross-Country Comparison
  • BCG Robotics Outlook 2030
  • 2021 World Bank Review of National AI Strategies and Policies
• US Roadmaps
  • 2020 US National Robotics Roadmap
  • ASME: Robotics & Covid 19
  • ASME: Accelerating US Robotics for American Prosperity and Security
  • 2020 NASA Technology Taxonomy
  • 2017 Automated Vehicles 3.0: Preparing for the Future of Transportation
  • NITRD Supplement to the President’s FY2021 Budget
  • American AI Initiative
• Canada Roadmaps
  • Report from Canada’s Economic Strategy Tables: Advanced Manufacturing
  • Roadmap to 2030: A path towards doubling Canadian Manufacturing and Output
  • CIFAR: Building an AI World
  • Pan-Canadian Artificial Intelligence Strategy
• Latam Roadmaps
  • AI for Social Good in Latin America and the Caribbean
  • Mexico Agenda National for Artificial Intelligence
  • Brazil Industry 4.0
  • Brazil E-Digital
  • Colombia National Policy for Digital Transformation and Artificial Intelligence
  • Chile National Artificial Intelligence Policy
  • Argentina Agenda Digital 2030
  • Uruguay Artificial Intelligence Strategy for the Public Administration
• UK & EU Roadmaps
  • AI in the UK: No Room for Complacency
  • UK’s Industrial Strategy
  • SPARC: Robotics 2020 Multi-Annual Roadmap for Robotics in Europe
  • SRIDA: AI, Data and Robotics Partnership
  • EC: Communication on Artificial Intelligence
  • Sweden National Approach for Artificial Intelligence
  • Spain RDI Strategy in Artificial Intelligence
  • Strategy for the Development of Artificial Intelligence in the Republic of Serbia for the period 2020-2025
  • AI Portugal 2030
  • Assumptions for the AI strategy in Poland
  • Norway National Strategy for Artificial Intelligence
  • The Declaration on AI in the Nordic-Baltic Region
  • Malta the Ultimate AI Launchpad: A Strategy and Vision for Artificial Intelligence in Malta 2030
  • Artificial Intelligence: a strategic vision for Luxembourg
  • Lithuanian Artificial Intelligence Strategy: A vision of the future
  • Italy National Strategy on Artificial Intelligence
  • Made in Germany: Artificial Intelligence Strategy
  • Meaningful Artificial Intelligence: Towards a French and European Strategy
  • Finland’s Age of Artificial Intelligence
  • Finland Leading the Way into the Age of Artificial Intelligence
  • Denmark National Strategy for Artificial Intelligence
  • Czech National Artificial Intelligence Strategy
  • Cyprus National Strategy AI
  • AI 4 Belgium
  • Austria AIM AT 2030
• Russia Roadmaps
  • National Strategy for the Development of Artificial Intelligence by 2030
• Africa & MENA Roadmaps
  • PIDA Closing the Infrastructure Gap Vital for Africa’s Transformation
  • Africa50
  • AIDA Accelerated Industrial Development for Africa
  • UAE Strategy for Artificial Intelligence
  • UAE Artificial Intelligence Strategy 2031
  • Saudi Arabia Vision 2030
  • National AI Strategy: Unlocking Tunisia’s capabilities potential
• Asia & SE Asia Roadmaps
  • 2017 A Robotics and Automation Roadmap for India
  • 2018 India National Strategy for Artificial Intelligence
  • AI Singapore
  • Korea AI Manufacturing 2030
• China Roadmaps
  • The 14th Five-Year Plan (2021-2025) for National Economic and Social Development of the People’s Republic of China
  • *Interpretation of “The “14th Five-Year” Robot Industry Development Plan”*
  • 2017 A New Generation Artificial Intelligence Development Plan
• Japan Roadmaps
  • 2021 Projects in Robotics and Artificial Intelligence
  • 2019 AI for Everyone: People, Industries, Regions and Governments
  • 2017 NEDO’s Activities in Robotics and Artificial Intelligence
  • 2015 METI Japan’s Robot Strategy
• Australia Roadmaps
  • A Robotics Roadmap for Australia
  • 2019 Artificial Intelligence Roadmap
  • Artificial Intelligence Standards Roadmap: Making Australia’s Voice Heard
  • 2017 The Automation Advantage
• New Zealand & Pacifica Roadmaps
  • New Zealand Robotics Automation and Sensing Roadmap
  • Towards our intelligent future: an AI roadmap for New Zealand
• Non Terrestrial Roadmaps

Do you know of robotics roadmaps not yet included? Please share them with us.

View Details

Robot and artificial intelligence are poised to increase their influences within our every day lives. (Shutterstock)

By Shane Saunderson

In the mid-1990s, there was research going on at Stanford University that would change the way we think about computers. The Media Equation experiments were simple: participants were asked to interact with a computer that acted socially for a few minutes after which, they were asked to give feedback about the interaction.

Participants would provide this feedback either on the same computer (No. 1) they had just been working on or on another computer (No. 2) across the room. The study found that participants responding on computer No. 2 were far more critical of computer No. 1 than those responding on the same machine they’d worked on.

People responding on the first computer seemed to not want to hurt the computer’s feelings to its face, but had no problem talking about it behind its back. This phenomenon became known as the computers as social actors (CASA) paradigm because it showed that people are hardwired to respond socially to technology that presents itself as even vaguely social.

The CASA phenomenon continues to be explored, particularly as our technologies have become more social. As a researcher, lecturer and all-around lover of robotics, I observe this phenomenon in my work every time someone thanks a robot, assigns it a gender or tries to justify its behaviour using human, or anthropomorphic, rationales.

What I’ve witnessed during my research is that while few are under any delusions that robots are people, we tend to defer to them just like we would another person.

Social tendencies While this may sound like the beginnings of a Black Mirror episode, this tendency is precisely what allows us to enjoy social interactions with robots and place them in caregiver, collaborator or companion roles.

The positive aspects of treating a robot like a person is precisely why roboticists design them as such — we like interacting with people. As these technologies become more human-like, they become more capable of influencing us. However, if we continue to follow the current path of robot and AI deployment, these technologies could emerge as far more dystopian than utopian.

The Sophia robot, manufactured by Hanson Robotics, has been on 60 Minutes, received honorary citizenship from Saudi Arabia, holds a title from the United Nations and has gone on a date with actor Will Smith. While Sophia undoubtedly highlights many technological advancements, few surpass Hanson’s achievements in marketing. If Sophia truly were a person, we would acknowledge its role as an influencer.

However, worse than robots or AI being sociopathic agents — goal-oriented without morality or human judgment — these technologies become tools of mass influence for whichever organization or individual controls them.

If you thought the Cambridge Analytica scandal was bad, imagine what Facebook’s algorithms of influence could do if they had an accompanying, human-like face. Or a thousand faces. Or a million. The true value of a persuasive technology is not in its cold, calculated efficiency, but its scale.

Seeing through intent Recent scandals and exposures in the tech world have left many of us feeling helpless against these corporate giants. Fortunately, many of these issues can be solved through transparency.

There are fundamental questions that are important for social technologies to answer because we would expect the same answers when interacting with another person, albeit often implicitly. Who owns or sets the mandate of this technology? What are its objectives? What approaches can it use? What data can it access?

Since robots could have the potential to soon leverage superhuman capabilities, enacting the will of an unseen owner, and without showing verbal or non-verbal cues that shed light on their intent, we must demand that these types of questions be answered explicitly.

As a roboticist, I get asked the question, “When will robots take over the world?” so often that I’ve developed a stock answer: “As soon as I tell them to.” However, my joke is underpinned by an important lesson: don’t scapegoat machines for decisions made by humans.

I consider myself a robot sympathizer because I think robots get unfairly blamed for many human decisions and errors. It is important that we periodically remind ourselves that a robot is not your friend, your enemy or anything in between. A robot is a tool, wielded by a person (however far removed), and increasingly used to influence us.

Shane receives funding from the Natural Sciences and Engineering Research Council of Canada (NSERC). He is affiliated with the Human Futures Institute, a Toronto-based think tank.

This article appeared in The Conversation.

View Details

Congratulations to Tao Chen, Jie Xu and Pulkit Agrawal who have won the CoRL 2021 best paper award!

Their work, A system for general in-hand object re-orientation, was highly praised by the judging committee who commented that “the sheer scope and variation across objects tested with this method, and the range of different policy architectures and approaches tested makes this paper extremely thorough in its analysis of this reorientation task”.

Below, the authors tell us more about their work, the methodology, and what they are planning next.

What is the topic of the research in your paper? We present a system for reorienting novel objects using an anthropomorphic robotic hand with any configuration, with the hand facing both upwards and downwards. We demonstrate the capability of reorienting over 2000 geometrically different objects in both cases. The learned controller can also reorient novel unseen objects.

Could you tell us about the implications of your research and why it is an interesting area for study? Our learned skill (in-hand object reorientation) can enable fast pick-and-place of objects in desired orientations and locations. For example, in logistics and manufacturing, it is a common demand to pack objects into slots for kitting. Currently, this is usually achieved via a two-stage process involving re-grasping. Our system will be able to achieve it in one step, which can substantially improve the packing speed and boost the manufacturing efficiency.

Another application is enabling robots to operate a wider variety of tools. The most common end-effector in industrial robots is a parallel-jaw gripper, partially due to its simplicity in control. However, such an end-effector is physically unable to handle many tools we see in our daily life. For example, even using pliers is difficult for such a gripper as it cannot dexterously move one handle back and forth. Our system will allow a multi-fingered hand to dexterously manipulate such tools, which opens up a new area for robotics applications.

Could you explain your methodology? We use a model-free reinforcement learning algorithm to train the controller for reorienting objects. In-hand object reorientation is a challenging contact-rich task. It requires a tremendous amount of training. To speed up the learning process, we first train the policy with privileged state information such as object velocities. Using the privileged state information drastically improves the learning speed. Other than this, we also found that providing a good initialization on the hand and object pose is critical for training the controller to reorient objects when the hand faces downward. In addition, we develop a technique to facilitate the training by building a curriculum on gravitational acceleration. We call this technique “gravity curriculum”.

With these techniques, we are able to train a controller that can reorient many objects even with a downward-facing hand. However, a practical concern of the learned controller is that it makes use of privileged state information, which can be nontrivial to get in the real world. For example, it is hard to measure the object’s velocity in the real world. To ensure that we can deploy a controller reliably in the real world, we use teacher-student training. We use the controller trained with the privileged state information as the teacher. Then we train a second controller (student) that does not rely on any privileged state information and hence has the potential to be deployed reliably in the real world. This student controller is trained to imitate the teacher controller using imitation learning. The training of the student controller becomes a supervised learning problem and is therefore sample-efficient. In the deployment time, we only need the student controller.

What were your main findings? We developed a general system that can be used to train controllers that can reorient objects with either the robotic hand facing upward or downward. The same system can also be used to train controllers that use external support such as a supporting surface for object re-orientation. Such controllers learned in our system are robust and can also reorient unseen novel objects. We also identified several techniques that are important for training a controller to reorient objects with a downward-facing hand.

A priori one might believe that it is important for the robot to know about object shape in order to manipulate new shapes. Surprisingly, we find that the robot can manipulate new objects without knowing their shape. It suggests that robust control strategies mitigate the need for complex perceptual processing. In other words, we might need much simpler perceptual processing strategies than previously thought for complex manipulation tasks.

What further work are you planning in this area? Our immediate next step is to achieve such manipulation skills on a real robotic hand. To achieve this, we will need to tackle many challenges. We will investigate overcoming the sim-to-real gap such that the simulation results can be transferred to the real world. We also plan to design new robotic hand hardware through collaboration such that the entire robotic system can be dexterous and low-cost.

About the authors Tao Chen is a Ph.D. student in the Improbable AI Lab at MIT CSAIL, advised by Professor Pulkit Agrawal. His research interests revolve around the intersection of robot learning, manipulation, locomotion, and navigation. More recently, he has been focusing on dexterous manipulation. His research papers have been published in top AI and robotics conferences. He received his master’s degree, advised by Professor Abhinav Gupta, from the Robotics Institute at CMU, and his bachelor’s degree from Shanghai Jiao Tong University.

Jie Xu is a Ph.D. student at MIT CSAIL, advised by Professor Wojciech Matusik in the Computational Design and Fabrication Group (CDFG). He obtained a bachelor’s degree from Department of Computer Science and Technology at Tsinghua University with honours in 2016. During his undergraduate period, he worked with Professor Shi-Min Hu in the Tsinghua Graphics & Geometric Computing Group. His research mainly focuses on the intersection of Robotics, Simulation, and Machine Learning. Specifically, he is interested in the following topics: robotics control, reinforcement learning, differentiable physics-based simulation, robotics control and design co-optimization, and sim-to-real.

Dr Pulkit Agrawal is the Steven and Renee Finn Chair Professor in the Department of Electrical Engineering and Computer Science at MIT. He earned his Ph.D. from UC Berkeley and co-founded SafelyYou Inc. His research interests span robotics, deep learning, computer vision and reinforcement learning. Pulkit completed his bachelor’s at IIT Kanpur and was awarded the Director’s Gold Medal. He is a recipient of the Sony Faculty Research Award, Salesforce Research Award, Amazon Machine Learning Research Award, Signatures Fellow Award, Fulbright Science and Technology Award, Goldman Sachs Global Leadership Award, OPJEMS, and Sridhar Memorial Prize, among others.

Find out more * Read the paper on arXiv. * The videos of the learned policies are available here, as is a video of the authors’ presentation at CoRL. * Read more about the winning and shortlisted papers for the CoRL awards here.

View Details

Kate speaks with Brad Bogolea, CEO and Co-founder of Simbe Robotics. Simbe Robotics developed a mobile robot named Tally, which is bringing advanced shelf insights to improve the retail shopping experience.

Tally provides a state-of-the-art sensing system on a robust, scalable platform that collects analytics in real-time.

Brad Bogolea

Brad Bogolea is the CEO and Co-Founder of Simbe Robotics, where he is responsible for the company’s vision and execution of its leading retail intelligence solution. In November 2015, Brad brought to market the Tally robot, the world’s first autonomous shelf auditing and analytics solution to help retailers ensure merchandise is always stocked, in the right place, and correctly priced. The National Retail Federation Foundation has named Brad to its list of “People Shaping Retail’s Future.

Prior to Simbe, Brad spent 10 years in the energy and wireless sensor industry. Most recently Brad worked at Silver Spring Networks, where he led global product management and business development efforts for their energy management and data analytics platform for energy utilities. Products under Brad’s leadership allowed the world’s largest utility companies and their energy consumers alike to gain efficiency and visibility of their energy usage through smart sensors and data.

Brad holds a B.S. in Computer Science and Engineering from Pennsylvania State University where he also returned to serve as an Entrepreneur-in-Residence.

Links

• Simbe Robotics
• Download mp3 (34.8 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

View Details

Sure the average video gamer is 34 years old, but the most active group is boys under 18, a group famously resistant to reading. Here is the RTSF Top 10 recommendations of books that have robots plus enough world building to rival Halo or Doom and lots of action or puzzles to solve. What’s even cooler is that you can cleverly use the “Topics” links to work in some STEM talking points by asking things like: do you think it would be easy reprogram cars to hit pedestrians instead of avoiding them? How would you fool a security drone? or Do you think robots should have the same rights as animals? But you may want to read them too, the first six on the list are books that I routinely recommend to general audiences and people tell me how much they loved them.

Head On – The rugby-like game in the book, Hilketa, played with real robots, is the best multiplayer game that never was. And paralyzed people have an advantage! (FYI: a PG-13 discussion of tele-sex through robots). Good for teachable moments about teleoperation.

Robopocalypse– Loved World War Z and read the book? They’ll love this more and it’s largely accurate about robots. Good for teachable moments about robot autonomy.

The Murderbot Diaries (series)- Delightfully snarky point of view of a security robot trying to save clueless scientists from Aliens-like corpos and creatures. Good for teachable moments about software engineering and whether intelligence systems would need a governor to keep them in line.

The Electric State– This is sort of a graphic novel the way Hannah Gadsby is sort of a comedian- it transcends the genre. Neither the full page illustrations nor the accompanying text tell the whole story of the angry teenage girl and her robot trying to outrun the end of the world. Like an escape room, you have to put the text and images together to figure out what is going on. Good for teachable moments about autonomy.

Tales from the Loop– the graphic novels, two in the series, are different from the emo Amazon streaming series. The books are much more suited to a teenage audience who love world building and surprising twists. Good for teachable moments about bounded rationality.

Kill Decision– Scarily realistic description of killer drones, with cool Spec Ops guy who has two ravens with call out to Norse mythology. Good for teachable moments about swarms (aka multi-robot systems).

Robots of Gotham– It’s sort of Game Lit without being based on a video game. Excellent discussion of how computer vision/machine learning works. Good for teachable moments about computer vision and machine learning.

The Andromeda Evolution– Helps if they’ve seen or read the original Andromeda Strain movie, but it can be read as a stand-alone. This commissioned sequel is a worthy addition. Good for teachable moments about drones and teleoperation.

Machinehood – A pro-Robots Rights group is terrorizing the world, nice discussion of ethics amid a lot of action- no boring lectures. Good for teachable moments about robot ethics.

The Themis Files– A earnest girl finds an alien Pacific Rim robot and learns to use it to fight evil giant piloted mecha invaders while shadowy quasi-governmental figures try to uncover its origins. Good for teachable moments about exoskeletons.

View Details

The Conference on Robot Learning (CoRL) is an annual international conference specialised in the intersection of robotics and machine learning. The fifth edition took place last week in London and virtually around the globe. Apart from the novelty of being a hybrid conference, this year the focus was put on openness. OpenReview was used for the peer review process, meaning that the reviewers’ comments and replies from the authors are public, for anyone to see. The research community suggests that open review could encourage mutual trust, respect, and openness to criticism, enable constructive and efficient quality assurance, increase transparency and accountability, facilitate wider, and more inclusive discussion, give reviewers recognition and make reviews citable [1]. You can access all CoRL 2021 papers and their corresponding reviews here. In addition, you may want to listen to all presentations, available in the conference YouTube channel.

In this post we bring you a glimpse of the conference through the most popular tweets written last week. Cool robot demos, short and sweet explanation of papers and award finalists to look forward to next year’s edition in New Zealand. Enjoy!

Robots, robots, robots!

Here’s our fourth and final #CoRL2021 paper, on multi-stage imitation learning: https://t.co/22JqLMMp4X

We’re doing a live demo of this right now, see below! This sequence was trained with a single demonstration, and generalises across poses / distractors.

With @normandipalo. pic.twitter.com/ENEzKvAlRm

— Edward Johns (@Ed__Johns) November 11, 2021

ANYmal trotting around at #CoRL2021 pic.twitter.com/59ql6sEJEx

— Kai Arulkumaran (@kaixhin) November 9, 2021

When I am human – I like dogs a lot. But it turns out that when I am a robot – I am a bit timid around quadrupeds. Thanks to the wonderful team of #CoRL2021 who organized telepresence sessions (@Ed__Johns, @vitalisvos19, Binbin Xu, and others). This was a very curious experience! pic.twitter.com/mMEdqUq6yv

— Rika Antonova (@contactrika) November 11, 2021

We will be doing a live demo of iMAP at #corl2021, sessions 1 and 8 Monday and Thursday, come if you're around!https://t.co/Tagk4jocbe@liu_shikun, @joeaortiz, @AjdDavison pic.twitter.com/XTUYrkOWnO

— Edgar Sucar (@SucarEdgar) November 8, 2021

Papers and presentations

Excited to present our work at #CoRL2021 this week. Our paper shows a promising approach to achieve sim2real transfer for perception. w/ @ToyotaResearch @berkeley_ai (1/7) https://t.co/MAbYaXaPrY pic.twitter.com/scfv4DtOSJ

— Michael laskey (@Michaellaskey7) November 8, 2021

Ever wondered how to tune your hyperparameters while training RL agents? w/o running thousands of experiments in parallel? And even combine them?

Check out our work @ #CoRL2021 on training mixture agents which combines components with diverse architectures, distributions, etc pic.twitter.com/4tsqWXjf79

— Markus Wulfmeier (@markus_with_k) November 9, 2021

Proud to share "LILA: Language-Informed Latent Actions," our paper at #CoRL2021.

How can we build assistive controllers by fusing language & shared autonomy?

Jointly authored w/ @megha_byte, with my advisors @percyliang & @DorsaSadigh.

: https://t.co/6vdprK9gX0

: (1 / 10) pic.twitter.com/K5vdUO8R6q

— Siddharth Karamcheti (@siddkaramcheti) November 8, 2021

In daily life after passing the paper exam for a permit, a newbie can drive the car for real under an expert's monitoring. This guardian mechanism is crucial for exploratory but safe learning. See our #CoRL2021 work https://t.co/OegFMwj9JS
Poster session today at Session 8 pic.twitter.com/IlKkZhk8UD

— Bolei Zhou (@zhoubolei) November 11, 2021

I've just finished my talk in the Blue Sky oral session at #CoRL2021. Great to have a session which allows for unconventional opinions to be heard in this way. Looking forward to the next two!

Here's my paper, Back to Reality for Imitation Learning: https://t.co/TZLj77zGWV. pic.twitter.com/NnwZNaSsX5

— Edward Johns (@Ed__Johns) November 9, 2021

This week at #CoRL2021, our team explores how data efficient off-policy RL, on-board reward generation, and vision can teach walking and wall avoidance to small humanoid robots without laboratory instrumentation: https://t.co/ZZo9hJXXO5 1/2 pic.twitter.com/4qWqn1hefF

— DeepMind (@DeepMind) November 10, 2021

Awards

Congratulations again to the #CoRL #BestSystemPaper Winner, "FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding", by Huy Ha and Shuran Song
Read it here: https://t.co/en8ic29gPM#robot #learning #robotics #award pic.twitter.com/P9ZW884dDn

— Conference on Robot Learning (@corl_conf) November 12, 2021

Congratulations again to the #CoRL #BestPaper Winner, "A System for General In-Hand Object Re-Orientation", by Tao Chen, Jie Xu and Pulkit Agrawal
Read it here: https://t.co/QeIzAW1fng#robot #learning #robotics #award pic.twitter.com/niHDCArbmq

— Conference on Robot Learning (@corl_conf) November 12, 2021

Congratulations to #CoRL2021 best paper finalist, "Robot Reinforcement Learning on the Constraint Manifold", Puze Liu, Davide Tateo, Haitham Bou Ammar, Jan Peters.https://t.co/SdssOu3akD#robotics #learning #award #research pic.twitter.com/Lpgn2XjEhY

— Conference on Robot Learning (@corl_conf) November 8, 2021

Congratulations to #CoRL2021 best paper finalist, "Learning Off-Policy with Online Planning", Harshit Sikchi, Wenxuan Zhou, David Held.https://t.co/7xUFyMlyQU#robotics #learning #award #research pic.twitter.com/D5u4NdDDST

— Conference on Robot Learning (@corl_conf) November 8, 2021

Congratulations to #CoRL2021 best paper finalist, "XIRL: Cross-embodiment Inverse Reinforcement Learning", Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, Debidatta Dwibedi.https://t.co/Z4qVMiuhgS#robotics #learning #award #research pic.twitter.com/k2KSkv67vd

— Conference on Robot Learning (@corl_conf) November 8, 2021

Congratulations to #CoRL2021 best systems paper finalist, "SORNet: Spatial Object-Centric Representations for Sequential Manipulation", Wentao Yuan, Chris Paxton, Karthik Desingh, Dieter Fox.https://t.co/en8ic29gPM#robotics #learning #award #research pic.twitter.com/Hfn1CiZNP8

— Conference on Robot Learning (@corl_conf) November 8, 2021

Congratulations to #CoRL2021 best systems paper finalist, "Fast and Efficient Locomotion via Learned Gait Transitions", Yuxiang Yang, Tingnan Zhang, Erwin Coumans, Jie Tan, Byron Boots.https://t.co/HbrCOVZOpz#robotics #learning #award #research pic.twitter.com/T5Ev31PgPZ

— Conference on Robot Learning (@corl_conf) November 8, 2021

References * PAPER – Ten considerations for open peer review. Schmidt B, Ross-Hellauer T, van Edig X, and Moylan EC. F1000Res. 2018;7:969. Published 2018 Jun 29. doi:10.12688/f1000research.15334.1

View Details

The new microbot inspired by starfish larva stirs up plastic beads. (Image: Cornel Dillinger/ETH Zurich)

By Rahel Künzler

Among scientists, there is great interest in tiny machines that are set to revolutionise medicine. These microrobots, often only a fraction of the diameter of a hair, are made to swim through the body to deliver medication to specific areas and perform the smallest surgical procedures.

The designs of these robots are often inspired by natural microorganisms such as bacteria or algae. Now, for the first time, a research group at ETH Zurich has developed a microrobot design inspired by starfish larva, which use ciliary bands on their surface to swim and feed. The ultrasound-​activated synthetic system mimics the natural arrangements of starfish ciliary bands and leverages nonlinear acoustics to replicate the larva’s motion and manipulation techniques.

Hairs to push liquid away or suck it in Depending on whether it is swimming or feeding, the starfish larva generates different patterns of vortices. (Image: Prakash Lab, Stanford University)

At first glance, the microrobots bear only scant similarity to starfish larva. In its larval stage, a starfish has a lobed body that measures just a few millimetres across. Meanwhile, the microrobot is a rectangle and ten times smaller, only a quarter of a millimetre across. But the two do share one important feature: a series of fine, movable hairs on the surface, called cilia.

A starfish larva is blanketed with hundreds of thousands of these hairs. Arranged in rows, they beat back and forth in a coordinated fashion, creating eddies in the surrounding water. The relative orientation of two rows determines the end result: Inclining two bands of beating cilia toward each other creates a vortex with a thrust effect, propelling the larva. On the other hand, inclining two bands away from each other creates a vortex that draws liquid in, trapping particles on which the larva feeds.

Artificial swimmers beat faster These cilia were the key design element for the new microrobot developed by ETH researchers led by Daniel Ahmed, who is a Professor of Acoustic Robotics for life sciences and healthcare. “In the beginning,” Ahmed said, “we simply wanted to test whether we could create vortices similar to those of the starfish larva with rows of cilia inclined toward or away from each other.

To this end, the researchers used photolithography to construct a microrobot with appropriately inclined ciliary bands. They then applied ultrasound waves from an external source to make the cilia oscillate. The synthetic versions beat back and forth more than ten thousand times per second – about a thousand times faster than those of a starfish larva. And as with the larva, these beating cilia can be used to generate a vortex with a suction effect at the front and a vortex with a thrust effect at the rear, the combined effect “rocketing” the robot forward.

Besides swimming, the new microrobot can collect particles and steer them in a predetermined direction. (Video: Cornel Dillinger/ETH Zurich)

In their lab, the researchers showed that the microrobots can swim in a straight line through liquid such as water. Adding tiny plastic beads to the water made it possible to visualize the vortices created by the microrobot. The result is astonishing: both starfish larva and microrobots generate virtually identical flow patterns.

Next, the researchers arranged the ciliary bands so that a suction vortex was positioned next to a thrust vortex, imitating the feeding technique used by starfish larva. This arrangement enabled the robots to collect particles and send them out in a predetermined direction.

Ultrasound offers many advantages Ahmed is convinced that this new type of microrobot will be ready for use in medicine in the foreseeable future. This is because a system that relies only on ultrasound offers decisive advantages: ultrasound waves are already widely used in imaging, penetrate deep inside the body, and pose no health risks.

“Our vision is to use ultrasound for propulsion, imaging and drug delivery.”

– Daniel Ahmed

The fact that this therapy requires only an ultrasound device makes it cheap, he adds, and hence suitable for use in both developed and developing countries.

Ahmed believes one initial field of application could be the treatment of gastric tumours. Uptake of conventional drugs by diffusion is inefficient, but having microrobots transport a drug specifically to the site of a stomach tumour and then deliver it there might make the drug’s uptake into tumour cells more efficient and reduce side effects.

Sharper images thanks to contrast agents But before this vision can be realized, a major challenge remains to be overcome: imaging. Steering the tiny machines to the right place requires that a sharp image be generated in real time. The researchers have plans to make the microrobots more visible by incorporating contrast agents such as those already used in medical imaging with ultrasound.

In addition to medical applications, Ahmed anticipates this starfish-​inspired design to have important implications for the manipulation of smallest liquid volumes in research and in industry. Bands of beating cilia could execute tasks such as mixing, pumping and particle trapping.

• PAPER – Ultrasound-​activated ciliary bands for microrobotic systems inspired by starfish. Dillinger C, Nama N and Ahmed D. Nat Commun. 2021 Nov 9. doi: 10.1038/s41467-​021-26607-ycall_made

This articled was originally published at the official website of ETH Zürich. Read the original article here.

View Details

In this episode, Audrow Nash speaks to Jason Richards, CEO at DaxBot. DaxBot makes a charismatic robot for food delivery. Jason speaks on their delivery robot, their crowdfunding campaign, DaxBot’s revenue model, working with lawmakers to have robots on the sidewalks, and Jason teases their realtime operating system, DaxOS.

Episode Links

• Download the episode
• Jason’s LinkedIn
• Daxbot’s website
• Daxbot’s crowdfunding campaign

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes on YouTube

View Details

By Marco Arruda

This is the second chapter of the series “Exploring ROS2 with a wheeled robot”. In this episode, you’ll learn how to subscribe to a ROS2 topic using ROS2 C++.

You’ll learn:

• How to create a node with ROS2 and C++
• How to subscribe to a topic with ROS2 and C++
• How to launch a ROS2 node using a launch file

1 – Setup environment – Launch simulation Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create a ROS2 node Our goal is to read the laser data, so create a new file called reading_laser.cpp:

touch ~/ros2_ws/src/my_package/reading_laser.cpp

And paste the content below:

```

include "rclcpp/rclcpp.hpp"

include "sensor_msgs/msg/laser_scan.hpp"

using std::placeholders::_1;

class ReadingLaser : public rclcpphttps://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/::Node {

public: ReadingLaser() : Node("reading_laser") {

auto default\_qos = rclcpp::QoS(rclcpp::SystemDefaultsQoS());

subscription\_ = this->create\_subscription(
    "laser\_scan", default\_qos,
    std::bind(&ReadingLaser::topic\_callback, this, \_1));

}

private: void topic_callback(const sensor_msgs::msg::LaserScan::SharedPtr _msg) { RCLCPP_INFO(this->get_logger(), "I heard: '%f' '%f'", _msg->ranges[0], _msg->ranges[100]); } rclcpp::Subscription::SharedPtr subscription_; };

int main(int argc, char *argv[]) { rclcpp::init(argc, argv); auto node = std::make_shared(); RCLCPP_INFO(node->get_logger(), "Hello my friends"); rclcpp::spin(node); rclcpp::shutdown(); return 0; } ```

We are creating a new class ReadingLaser that represents the node (it inherits rclcpp::Node). The most important about that class are the subscriber attribute and the method callback. In the main function we are initializing the node and keep it alive (spin) while its ROS connection is valid.

The subscriber constructor expects to get a QoS, that stands for the middleware used for the quality of service. You can have more information about it in the reference attached, but in this post we are just using the default QoS provided. Keep in mind the following parameters:

• topic name
• callback method

The callback method needs to be binded, which means it will not be execute at the subscriber declaration, but when the callback is called. So we pass the reference of the method and setup the this reference for the current object to be used as callback, afterall the method itself is a generic implementationhttps://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/ of a class.

3 – Compile and run In order to compile the cpp file, we must add some instructions to the ~/ros2_ws/src/my_package/src/CMakeLists.txt:

• Look for find dependencies and include the sensor_msgs library
• Just before the install instruction add the executable and target its dependencies
• Append another install instruction for the new executable we’ve just created

```

find dependencies

find_package(ament_cmake REQUIRED) find_package(rclcpp REQUIRED) find_package(sensor_msgs REQUIRED) ...

... add_executable(reading_laser src/reading_laser.cpp) ament_target_dependencies(reading_laser rclcpp std_msgs sensor_msgs) ...

... install(TARGETS reading_laser DESTINATION lib/${PROJECT_NAME}/ ) ```

Compile it:

colcon build --symlink-install --packages-select my_package

4 – Run the node and mapping the topic In order to run the executable created, you can use:

ros2 run my_package reading_laser

Although the the laser values won’t show up. That’s because we have a “hard coded” topic name laser_scan. No problem at all, when we can map topics using launch files. Create a new launch file ~/ros2_ws/src/my_package/launch/reading_laser.py:

``` from launch import LaunchDescription from launch_ros.actions import Node

def generate_launch_description():

reading\_laser = Node(
    package='my\_package',
    executable='reading\_laser',https://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/
    output='screen',
    remappings=[
        ('laser\_scan', '/dolly/laser\_scan')
    ]
)

return LaunchDescription([
    reading\_laser
])

```

In this launch file there is an instance of a node getting the executable as argument and it is setup the remappings attribute in order to remap from laser_scan to /dolly/laser_scan.

Run the same node using the launch file this time:

ros2 launch my_package reading_laser.launch.py

Add some obstacles to the world and the result must be similar to:

Related courses & extra links: * ROS2 Basics for C++ * Copy my rosject here * Dolly robot – Official repo

The post Exploring ROS2 with wheeled robot – #2 – How to subscribe to ROS2 laser scan topic appeared first on The Construct.

View Details

An example of our method deployed on a Clearpath Jackal ground robot (left) exploring a suburban environment to find a visual target (inset). (Right) Egocentric observations of the robot.

Imagine you’re in an unfamiliar neighborhood with no house numbers and I give you a photo that I took a few days ago of my house, which is not too far away. If you tried to find my house, you might follow the streets and go around the block looking for it. You might take a few wrong turns at first, but eventually you would locate my house. In the process, you would end up with a mental map of my neighborhood. The next time you’re visiting, you will likely be able to navigate to my house right away, without taking any wrong turns.

Such exploration and navigation behavior is easy for humans. What would it take for a robotic learning algorithm to enable this kind of intuitive navigation capability? To build a robot capable of exploring and navigating like this, we need to learn from diverse prior datasets in the real world. While it’s possible to collect a large amount of data from demonstrations, or even with randomized exploration, learning meaningful exploration and navigation behavior from this data can be challenging – the robot needs to generalize to unseen neighborhoods, recognize visual and dynamical similarities across scenes, and learn a representation of visual observations that is robust to distractors like weather conditions and obstacles. Since such factors can be hard to model and transfer from simulated environments, we tackle these problems by teaching the robot to explore using only real-world data.

Formally, we studied the problem of goal-directed exploration for visual navigation in novel environments. A robot is tasked with navigating to a goal location , specified by an image taken at . Our method uses an offline dataset of trajectories, over 40 hours of interactions in the real-world, to learn navigational affordances and builds a compressed representation of perceptual inputs. We deploy our method on a mobile robotic system in industrial and recreational outdoor areas around the city of Berkeley. RECON can discover a new goal in a previously unexplored environment in under 10 minutes, and in the process build a “mental map” of that environment that allows it to then reach goals again in just 20 seconds. Additionally, we make this real-world offline dataset publicly available for use in future research.

Rapid Exploration Controllers for Outcome-driven Navigation RECON, or Rapid Exploration Controllers for Outcome-driven Navigation, explores new environments by “imagining” potential goal images and attempting to reach them. This exploration allows RECON to incrementally gather information about the new environment.

Our method consists of two components that enable it to explore new environments. The first component is a learned representation of goals. This representation ignores task-irrelevant distractors, allowing the agent to quickly adapt to novel settings. The second component is a topological graph. Our method learns both components using datasets or real-world robot interactions gathered in prior work. Leveraging such large datasets allows our method to generalize to new environments and scale beyond the original dataset.

Learning to Represent Goals A useful strategy to learn complex goal-reaching behavior in an unsupervised manner is for an agent to set its own goals, based on its capabilities, and attempt to reach them. In fact, humans are very proficient at setting abstract goals for themselves in an effort to learn diverse skills. Recent progress in reinforcement learning and robotics has also shown that teaching agents to set its own goals by “imagining” them can result in learning of impressive unsupervised goal-reaching skills. To be able to “imagine”, or sample, such goals, we need to build a prior distribution over the goals seen during training.

For our case, where goals are represented by high-dimensional images, how should we sample goals for exploration? Instead of explicitly sampling goal images, we instead have the agent learn a compact representation of latent goals, allowing us to perform exploration by sampling new latent goal representations, rather than by sampling images. This representation of goals is learned from context-goal pairs previously seen by the robot. We use a variational information bottleneck to learn these representations because it provides two important properties. First, it learns representations that throw away irrelevant information, such as lighting and pixel noise. Second, the variational information bottleneck packs the representations together so that they look like a chosen prior distribution. This is useful because we can then sample imaginary representations by sampling from this prior distribution.

The architecture for learning a prior distribution for these representations is shown below. As the encoder and decoder are conditioned on the context, the representation only encodes information about relative location of the goal from the context – this allows the model to represent feasible goals. If, instead, we had a typical VAE (in which the input images are autoencoded), the samples from the prior over these representations would not necessarily represent goals that are reachable from the current state. This distinction is crucial when exploring new environments, where most states from the training environments are not valid goals.

The architecture for learning a prior over goals in RECON. The context-conditioned embedding learns to represent feasible goals.

To understand the importance of learning this representation, we run a simple experiment where the robot is asked to explore in an undirected manner starting from the yellow circle in the figure below. We find that sampling representations from the learned prior greatly accelerates the diversity of exploration trajectories and allows a wider area to be explored. In the absence of a prior over previously seen goals, using random actions to explore the environment can be quite inefficient. Sampling from the prior distribution and attempting to reach these “imagined” goals allows RECON to explore the environment efficiently.

Sampling from a learned prior allows the robot to explore 5 times faster than using random actions.

Goal-Directed Exploration with a Topological Memory We combine this goal sampling scheme with a topological memory to incrementally build a “mental map” of the new environment. This map provides an estimate of the exploration frontier as well as guidance for subsequent exploration. In a new environment, RECON encourages the robot to explore at the frontier of the map – while the robot is not at the frontier, RECON directs it to navigate to a previously seen subgoal at the frontier of the map.

At the frontier, RECON uses the learned goal representation to learn a prior over goals it can reliably navigate to and are thus, feasible to reach. RECON uses this goal representation to sample, or “imagine”, a feasible goal that helps it explore the environment. This effectively means that, when placed in a new environment, if RECON does not know where the target is, it “imagines” a suitable subgoal that it can drive towards to explore and collects information, until it believes it can reach the target goal image. This allows RECON to “search” for the goal in an unknown environment, all the while building up its mental map. Note that the objective of the topological graph is to build a compact map of the environment and encourage the robot to reach the frontier; it does not inform goal sampling once the robot is at the frontier.

Illustration of the exploration algorithm of RECON.

Learning from Diverse Real-world Data We train these models in RECON entirely using offline data collected in a diverse range of outdoor environments. Interestingly, we were able to train this model using data collected for two independent projects in the fall of 2019 and spring of 2020, and were successful in deploying the model to explore novel environments and navigate to goals during late 2020 and the spring of 2021. This offline dataset of trajectories consists of over 40 hours of data, including off-road navigation, driving through parks in Berkeley and Oakland, parking lots, sidewalks and more, and is an excellent example of noisy real-world data with visual distractors like lighting, seasons (rain, twilight etc.), dynamic obstacles etc. The dataset consists of a mixture of teleoperated trajectories (2-3 hours) and open-loop safety controllers programmed to collect random data in a self-supervised manner. This dataset presents an exciting benchmark for robotic learning in real-world environments due to the challenges posed by offline learning of control, representation learning from high-dimensional visual observations, generalization to out-of-distribution environments and test-time adaptation.

We are releasing this dataset publicly to support future research in machine learning from real-world interaction datasets, check out the dataset page for more information.

We train from diverse offline data (top) and test in new environments (bottom).

RECON in Action Putting these components together, let’s see how RECON performs when deployed in a park near Berkeley. Note that the robot has never seen images from this park before. We placed the robot in a corner of the park and provided a target image of a white cabin door. In the animation below, we see RECON exploring and successfully finding the desired goal. “Run 1” corresponds to the exploration process in a novel environment, guided by a user-specified target image on the left. After it finds the goal, RECON uses the mental map to distill its experience in the environment to find the shortest path for subsequent traversals. In “Run 2”, RECON follows this path to navigate directly to the goal without looking around.

In “Run 1”, RECON explores a new environment and builds a topological mental map. In “Run 2”, it uses this mental map to quickly navigate to a user-specified goal in the environment.

An illustration of this two-step process from an overhead view is show below, showing the paths taken by the robot in subsequent traversals of the environment:

(Left) The goal specified by the user. (Right) The path taken by the robot when exploring for the first time (shown in cyan) to build a mental map with nodes (shown in white), and the path it takes when revisiting the same goal using the mental map (shown in red).

Deploying in Novel Environments To evaluate the performance of RECON in novel environments, study its behavior under a range of perturbations and understand the contributions of its components, we run extensive real-world experiments in the hills of Berkeley and Richmond, which have a diverse terrain and a wide variety of testing environments.

We compare RECON to five baselines – RND, InfoBot, Active Neural SLAM, ViNG and Episodic Curiosity – each trained on the same offline trajectory dataset as our method, and fine-tuned in the target environment with online interaction. Note that this data is collected from past environments and contains no data from the target environment. The figure below shows the trajectories taken by the different methods for one such environment.

We find that only RECON (and a variant) is able to successfully discover the goal in over 30 minutes of exploration, while all other baselines result in collision (see figure for an overhead visualization). We visualize successful trajectories discovered by RECON in four other environments below.

(Top) When comparing to other baselines, only RECON is able to successfully find the goal. (Bottom) Trajectories to goals in four other environments discovered by RECON.

Quantitatively, we observe that our method finds goals over 50% faster than the best prior method; after discovering the goal and building a topological map of the environment, it can navigate to goals in that environment over 25% faster than the best alternative method.

Quantitative results in novel environments. RECON outperforms all baselines by over 50%.

Exploring Non-Stationary Environments One of the important challenges in designing real-world robotic navigation systems is handling differences between training scenarios and testing scenarios. Typically, systems are developed in well-controlled environments, but are deployed in less structured environments. Further, the environments where robots are deployed often change over time, so tuning a system to perform well on a cloudy day might degrade performance on a sunny day. RECON uses explicit representation learning in attempts to handle this sort of non-stationary dynamics.

Our final experiment tested how changes in the environment affected the performance of RECON. We first had RECON explore a new “junkyard” to learn to reach a blue dumpster. Then, without any more supervision or exploration, we evaluated the learned policy when presented with previously unseen obstacles (trash cans, traffic cones, a car) and weather conditions (sunny, overcast, twilight). As shown below, RECON is able to successfully navigate to the goal in these scenarios, showing that the learned representations are invariant to visual distractors that do not affect the robot’s decisions to reach the goal.

First-person videos of RECON successfully navigating to a “blue dumpster” in the presence of novel obstacles (above) and varying weather conditions (below).

What’s Next? The problem setup studied in this paper – using past experience to accelerate learning in a new environment – is reflective of several real-world robotics scenarios. RECON provides a robust way to solve this problem by using a combination of goal sampling and topological memory.

A mobile robot capable of reliably exploring and visually observing real-world environments can be a great tool for a wide variety of useful applications such as search and rescue, inspecting large offices or warehouses, finding leaks in oil pipelines or making rounds at a hospital, delivering mail in suburban communities. We demonstrated simplified versions of such applications in an earlier project, where the robot has prior experience in the deployment environment; RECON enables these results to scale beyond the training set of environments and results in a truly open-world learning system that can adapt to novel environments on deployment.

We are also releasing the aforementioned offline trajectory dataset, with hours of real-world interaction of a mobile ground robot in a variety of outdoor environments. We hope that this dataset can support future research in machine learning using real-world data for visual navigation applications. The dataset is also a rich source of sequential data from a multitude of sensors and can be used to test sequence prediction models including, but not limited to, video prediction, LiDAR, GPS etc. More information about the dataset can be found in the full-text article.

This blog post is based on our paper Rapid Exploration for Open-World Navigation with Latent Goal Models, which will be presented as an Oral Talk at the 5th Annual Conference on Robot Learning in London, UK on November 8-11, 2021. You can find more information about our results and the dataset release on the project page.

Big thanks to Sergey Levine and Benjamin Eysenbach for helpful comments on an earlier draft of this article.

View Details

The market for professional service robots reached a turnover of 6.7 billion U.S. dollars worldwide (sample method) – up 12% in 2020. At the same time, turnover of new consumer service robots grew 16% to 4.4 billion U.S. dollars. This is according to World Robotics 2021 – Service Robots report, presented by the International Federation of Robotics (IFR).

TOP 5 Application in Service Robotics © World Robotics

“Service robots continued on a successful path proving the tremendous market potential worldwide,” says IFR President Milton Guerry. “Sales of professional service robots rose an impressive 41% to 131,800 units in 2020.”

Five top application trends for professional service robots were driven by extra demand of the global pandemic:

© IFR International Federation of Robotics

One out of three units were built for the transportation of goods or cargo. Turnover for Autonomous Mobile Robots (AMR) and delivery robots grew by 11% to over 1 billion US dollars. Most units sold operate in indoor environments for production and warehouses. The trend goes towards flexible solutions, so that the AMR´s act in mixed environments together e.g. with forklifts, other mobile robots or humans. There is also a strong market potential for transportation robots in outdoor environments with public traffic, e.g. lastmile delivery. Marketing and monetarization options will depend on the availability of regulatory frameworks which currently still prevent the large-scale deployment of such robots in most countries.

Demand for professional cleaning robots grew by 92% to 34,400 units sold. In response to increasing hygiene requirements due to the Covid-19 pandemic, more than 50 service robot providers developed disinfection robots, spraying disinfectant fluids, or using ultraviolet light. Often, existing mobile robots were modified to serve as disinfection robots. There is a high ongoing potential for disinfection robots in hospitals and other public places. Unit sales of professional floor cleaning robots are expected to grow by double-digit rates on average each year from 2021 to 2024.

In terms of value, the sales of medical robotics accounts for 55% of the total professional service robot turnover in 2020. This was mainly driven by robotic surgery devices, which are the most expensive type in the segment. Turnover increased by 11% to 3.6 billion U.S. dollars.

A tremendously growing number of robots for rehabilitation and non-invasive therapy make this application the largest medical one in terms of units. About 75% of medical robot suppliers are from North America and Europe.

The global pandemic created additional demand for social robots. They help e.g. residents of nursing homes to keep contact with friends and family members in times of social distancing. Communication robots provide information in public environments to avoid personal human contact, connect people via video for a business conference or help with maintanance tasks on the shopfloor.

Hospitality robots enjoy growing popularity generating turnover of 249 million US dollars. Demand for robots for food and drink preparation grew tremendously – turnover almost tripled to 32 million US dollars (+196%). The Covid-19 pandemic created increased awareness to avoid contact with food products. There is still a huge potential for hospitality robots with medium double-digit annual growth predicted.

Service robots for consumer use Robots for domestic tasks are the largest group of consumer robots. Almost 18.5 million units (+6%), worth 4.3 billion US dollars, were sold in 2020.

Robot vacuums and other robots for indoor domestic floor cleaning were up 5% to more than 17.2 million units with a value of 2.4 billion US dollar. This kind of service robot is available in almost every convenience store, making it easily accessible for everyone. Many American, Asian, and European suppliers cater to this market.

Gardening robots usually comprise lawn mowing robots. This market is expected to grow by low double-digit growth rates on average each year in the next few years.

Service robotics industry structure “The service robot industry is developing at a high pace,“ says IFR President Milton Guerry.” “Lots of start-up companies appear every year, developing innovative service robot applications and improving existing concepts. Some of these young companies disappear as quickly as they emerged. The activity remained high in the service robotics space with acquisitions by incumbents and acquisitions by companies from industries with a desire to expand and work in this exciting area.”

Company structure of service robot manufacturers © World Robotics 2021

Worldwide, 80% of the 1.050 service robot suppliers are considered incumbents that were established more than five years ago. 47% of the service robot suppliers are from Europe, 27% from North America and 25% from Asia.

World Robotics 2021 edition Orders for World Robotics 2021 Industrial Robots and Service Robots reports can be placed online. Further downloads on the content are available here.

Downloads Graphs, presentations and German press release are available below:

• Presentation World Robotics press conference extended version (5.9MB)
• Presentation World Robotics press conference short version (8.5MB)
• Graph: TOP 5 applications for professional use (98KB)
• Graph: Service Robot company structure (90KB)
• Overview: TOP Five Service Robot Trends 2021 (202KB)
• Pressemeldung Service Robots auf deutsch (173KB)

View Details

By Marco Arruda

This is the first chapter of the series “Exploring ROS2 with a wheeled robot”. In this episode, we setup our first ROS2 simulation using Gazebo 11. From cloning, compiling and creating a package + launch file to start the simulation!

You’ll learn:

• How to Launch a simulation using ROS2
• How to Compile ROS2 packages
• How to Create launch files with ROS2

1 – Start the environment In this series we are using ROS2 foxy, go to this page, create a new rosject selecting ROS2 Foxy distro and and run it.

2 – Clone and compile the simulation The first step is to clone the dolly robot package. Open a web shell and execute the following:

`cd ~/ros2_ws/src/

git clone https://github.com/chapulina/dolly.git`

Source the ROS 2 installation folder and compile the workspace:

`source /opt/ros/foxy/setup.bash

cd ~/ros2_ws

colcon build --symlink-install --packages-ignore dolly_ignition`

Notice we are ignoring the ignition related package, that’s because we will work only with gazebo simulator.

3 – Create a new package and launch file In order to launch the simulation, we will create the launch file from the scratch. It goes like:

`cd ~/ros2_ws/src

ros2 pkg create my_package --build-type ament_cmake --dependencies rclcpp`

After that, you must have the new folder my_package in your workspace. Create a new folder to contain launch files and the new launch file as well:

`mkdir -p ~/ros2_ws/src/my_package/launch

touch ~/ros2_ws/src/my_package/launch/dolly.launch.py`

Copy and paste the following to the new launch file:

``` import os

from ament_index_python.packages import get_package_share_directory from launch import LaunchDescription from launch.actions import DeclareLaunchArgument from launch.actions import IncludeLaunchDescription from launch.launch_description_sources import PythonLaunchDescriptionSource

def generate_launch_description():

pkg\_gazebo\_ros = get\_package\_share\_directory('gazebo\_ros')
pkg\_dolly\_gazebo = get\_package\_share\_directory('dolly\_gazebo')

gazebo = IncludeLaunchDescription(
    PythonLaunchDescriptionSource(
        os.path.join(pkg\_gazebo\_ros, 'launch', 'gazebo.launch.py')
    )
)

return LaunchDescription([
    DeclareLaunchArgument(
        'world',
        default\_value=[
            os.path.join(pkg\_dolly\_gazebo, 'worlds', 'dolly\_empty.world'), ''
        ],
        description='SDF world file',
    ),
    gazebo
])

```

Notice that a launch file returns a LaunchDescription that contains nodes or other launch files.

In this case, we have just included another launch file gazebo.launch.py and changed one of its arguments, the one that stands for the world name: world.

The robot, in that case, is included in the world file, so there is no need to have an extra spawn node, for example.

And append to the end of the file ~/ros2_ws/src/my_package/CMakeLists.txt the following instruction to install the new launch file into the ROS 2 environment:

``` install(DIRECTORY launch DESTINATION share/${PROJECT_NAME}/ ) ament_package()

```

4 – Compile and launch the simulation Use the command below to compile only the created package:

`cd ~/ros2_ws/

colcon build --symlink-install --packages-select my_package

source ~/ros2_ws/install/setup.bash

ros2 launch my_package dolly.launch.py`

5 – Conclusion This is how you can launch a simulation in ROS2. It is important to notice that:

• We are using a pre-made simulation: world + robot
• This is how a launch file is created: A python script
• In ROS2, you still have the same freedom of including other files or running executables inside a custom launch file

Related courses & extra links: * ROS2 Basics for C++ * Copy my rosject here * Dolly robot – Official repo

The post Exploring ROS2 with wheeled robot – #1 – Launch ROS2 Simulation appeared first on The Construct.

View Details

In this episode, Audrow Nash speaks to Tobias Holmes, Quality Assurance Manager at Blue River Technologies. Blue River uses computer vision and robotics in agriculture and was acquired by John Deere in 2017. Tobias speaks about herbicide resistance, spraying weeds, quality assurance and testing on hardware, and on encouraging kids to learn robotics.

Episode Links

• Download the episode
• Tobias’ LinkedIn
• Blue River Technology
• FIRST Robotics
• Crystal Ray High School (East Bay)

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes on YouTube

View Details

Scicomm.io is a science communication project which aims to empower people to share stories about their robotics and AI work. The project is a joint effort from Robohub and AIhub, both of which are educational platforms dedicated to connecting the robotics and AI communities to the rest of the world.

This project focuses on training the next generation of communicators in robotics and AI to build a strong connection with the outside world, by providing effective communication tools.

People working in the field are developing an enormous array of systems and technologies. However, due to a relative lack of high quality, impartial information in the mainstream media, the general public receive a lot hyped news which ends up causing fear and / or unrealistic expectations surrounding these technologies.

Scicomm.io has been created to facilitate the connection between the robotics and AI world and the rest of the world through teaching how to establish truthful, honest and hype-free communication. One that brings benefit to both sides.

Scicomm bytes With our series of bite-sized videos you can quickly learn about science communication for robotics and AI. Find out why science communication is important, how to talk to the media, and about some of the different ways in which you can communicate your work. We have also produced guides with tips for turning your research into blog post and for avoiding hype when promoting your research.

Training Training the next generation of science communicators is an important mission for scicomm.io (and indeed Robohub and AIhub). As part of scicomm.io, we run training courses to empower researchers to communicate about their work. When done well, stories about AI and robotics can help increase the visibility and impact of the work, lead to new connections, and even raise funds. However, most researchers don’t engage in science communication, due to a lack of skills, time, and reach that makes the effort worthwhile.

With our workshops we aim to overcome these barriers and make communicating robotics and AI ‘easy’. This is done through short training sessions with experts, and hands-on practical exercises to help students begin their science communication journey with confidence.

A virtual scicomm workshop in action.

During the workshops, participants will hear why science communication matters, learn the basic techniques of science communication, build a story around their own research, and find out how to connect with journalists and other communicators. We’ll also discuss different science communication media, how to use social media, how to prepare blog posts, videos and press releases, how to avoid hype, and how to communicate work to a general audience.

For more information about our workshops, contact the team by email.

Find out more about the scicomm.io project here.

View Details

Proteus and sunset with the Swiss mountains in the background at Lake Thun (photo credit: Gallus Kaufmann)

In 2018, Christian Engler felt he’d studied enough theory at the ETH Zurich and longed to put it all into practice. It was evident to Christian that the best way to get hands-on experience was to start something himself. Others were not so sure. Especially when they heard about his ambition to revive a project from high school.

The project involved underwater robots, also known as Remotely Operated Vehicles (ROVs). But now Christian wanted to step it up a gear. He not only wanted to build an ROV, but he also wanted to take part at the international MATE ROV Competition 2019 – the biggest ROV competition in the world. His passion motivated a further seven students and Tethys Robotics was born.

Start of the journey To be able to compete at a high level, Tethys Robotics needed support. The main supporter was and still is Professor Siegwart and his Autonomous Systems Lab (ASL). Due to a previous project, SCUBO, the team had the chance to benefit from their experience and were able to use their carbon fibre shell to build their version of the underwater robot, the SCUBO 2.0.

SCUBO taking a sunbath at Lake Zurich (photo credit: Gallus Kaufmann)

While the original SCUBO was developed to film coral reefs, the new version of SCUBO had to be adapted to fulfil the tasks at the MATE ROV Competition. To prevent damaging the coral reefs, the original SCUBO had a different position and orientation of the eight actuators (also known as thrusters) at the cost of stability and control. Furthermore, no batteries were allowed at the competition inside of the robot. For that reason, the entire electronics, software interface and controlling had to be redesigned to be able to take part at the competition.

Essentially, the sensors of both SCUBO versions are the same consisting of Attitude and Heading Reference System (AHRS), pressure sensor, stereo camera and temperature sensor.

The only downside was that the ASL did not have enough space to accommodate us. Therefore, we ended up working in the lab’s basement. But this did not affect our motivation – now complete with ‘garage start-up’ vibe – or the successful outcome. Tethys Robotics was the first Swiss team ever to compete. Out of 75 teams, Tethys came in at 9th place.

Real world applications After the competition, SCUBO 2.0 was showcased at various exhibitions. This is how we came across the divers from the Swiss Explosive Ordnance Disposal (EOD), whose mission is to retrieve lost ammunition from Swiss lakes. Since this is a dangerous and very challenging task for divers, Tethys started collaborating on an underwater robot up to the task. The robot had to be modular, lightweight, and easily deployable. With the support of the ASL, the algorithms developed for drones have been implemented to the underwater drones of Tethys Robotics. And by having multiple tests at the operation site with the EOD divers and blank ammunition, the new underwater robot Proteus was developed with a real world application in mind. With their help, Proteus has been developed to be modular and more focused to be used as a diver buddy instead of working individually. In particular, the underwater drone can be used as a lift for material and as a communication device to the top side.

Challenges and future of Proteus There have been many changes to the design and the application field between the competition robot and the new Proteus. Proteus is able to be deployed in every Swiss lake (+300 m depth rating) and is powered again, as the original SCUBO, by batteries. The communication and camera streams are transmitted via a fiber optics cable to a control station on the shore. Moreover, in addition to the sensors used in SCUBO, Proteus has been upgraded with sensors mainly used for localization and orientation underwater which consist of a sonar, Doppler Velocity Logger (DVL), acoustic Short Baseline system (SBL) and event cameras.

Proteus mission setup (photo credit: Gallus Kaufmann)

The current research focuses on the improvement of the underwater localization. The different sensors are not accurate, fast or robust enough to be used in Swiss lakes and in particular rivers. Therefore, a sensor fusion algorithm is being implemented to combine the advantages of the different sensors. Furthermore, Tethys Robotics is trying to find further applications and an appropriate market segment to position the developed underwater drones. By finding further partners and interest, the project could make the transition to a company and develop the prototypes to final products.

View Details

Diagram of MURAL, our method for learning uncertainty-aware rewards for RL. After the user provides a few examples of desired outcomes, MURAL automatically infers a reward function that takes into account these examples and the agent’s uncertainty for each state.

Although reinforcement learning has shown success in domains such as robotics, chip placement and playing video games, it is usually intractable in its most general form. In particular, deciding when and how to visit new states in the hopes of learning more about the environment can be challenging, especially when the reward signal is uninformative. These questions of reward specification and exploration are closely connected — the more directed and “well shaped” a reward function is, the easier the problem of exploration becomes. The answer to the question of how to explore most effectively is likely to be closely informed by the particular choice of how we specify rewards.

For unstructured problem settings such as robotic manipulation and navigation — areas where RL holds substantial promise for enabling better real-world intelligent agents — reward specification is often the key factor preventing us from tackling more difficult tasks. The challenge of effective reward specification is two-fold: we require reward functions that can be specified in the real world without significantly instrumenting the environment, but also effectively guide the agent to solve difficult exploration problems. In our recent work, we address this challenge by designing a reward specification technique that naturally incentivizes exploration and enables agents to explore environments in a directed way.

Outcome Driven RL and Classifier Based Rewards While RL in its most general form can be quite difficult to tackle, we can consider a more controlled set of subproblems which are more tractable while still encompassing a significant set of interesting problems. In particular, we consider a subclass of problems which has been referred to as outcome driven RL. In outcome driven RL problems, the agent is not simply tasked with exploring the environment until it chances upon reward, but instead is provided with examples of successful outcomes in the environment. These successful outcomes can then be used to infer a suitable reward function that can be optimized to solve the desired problems in new scenarios.

More concretely, in outcome driven RL problems, a human supervisor first provides a set of successful outcome examples , representing states in which the desired task has been accomplished. Given these outcome examples, a suitable reward function can be inferred that encourages an agent to achieve the desired outcome examples. In many ways, this problem is analogous to that of inverse reinforcement learning, but only requires examples of successful states rather than full expert demonstrations.

When thinking about how to actually infer the desired reward function from successful outcome examples , the simplest technique that comes to mind is to simply treat the reward inference problem as a classification problem – “Is the current state a successful outcome or not?” Prior work has implemented this intuition, inferring rewards by training a simple binary classifier to distinguish whether a particular state is a successful outcome or not, using the set of provided goal states as positives, and all on-policy samples as negatives. The algorithm then assigns rewards to a particular state using the success probabilities from the classifier. This has been shown to have a close connection to the framework of inverse reinforcement learning.

Classifier-based methods provide a much more intuitive way to specify desired outcomes, removing the need for hand-designed reward functions or demonstrations:

These classifier-based methods have achieved promising results on robotics tasks such as fabric placement, mug pushing, bead and screw manipulation, and more. However, these successes tend to be limited to simple shorter-horizon tasks, where relatively little exploration is required to find the goal.

What’s Missing? Standard success classifiers in RL suffer from the key issue of overconfidence, which prevents them from providing useful shaping for hard exploration tasks. To understand why, let’s consider a toy 2D maze environment where the agent must navigate in a zigzag path from the top left to the bottom right corner. During training, classifier-based methods would label all on-policy states as negatives and user-provided outcome examples as positives. A typical neural network classifier would easily assign success probabilities of 0 to all visited states, resulting in uninformative rewards in the intermediate stages when the goal has not been reached.

Since such rewards would not be useful for guiding the agent in any particular direction, prior works tend to regularize their classifiers using methods like weight decay or mixup, which allow for more smoothly increasing rewards as we approach the successful outcome states. However, while this works on many shorter-horizon tasks, such methods can actually produce very misleading rewards. For example, on the 2D maze, a regularized classifier would assign relatively high rewards to states on the opposite side of the wall from the true goal, since they are close to the goal in x-y space. This causes the agent to get stuck in a local optima, never bothering to explore beyond the final wall!

In fact, this is exactly what happens in practice:

Uncertainty-Aware Rewards through CNML As discussed above, the key issue with unregularized success classifiers for RL is overconfidence — by immediately assigning rewards of 0 to all visited states, we close off many paths that might eventually lead to the goal. Ideally, we would like our classifier to have an appropriate notion of uncertainty when outputting success probabilities, so that we can avoid excessively low rewards without suffering from the misleading local optima that result from regularization.

Conditional Normalized Maximum Likelihood (CNML)

One method particularly well-suited for this task is Conditional Normalized Maximum Likelihood (CNML). The concept of normalized maximum likelihood (NML) has typically been used in the Bayesian inference literature for model selection, to implement the minimum description length principle. In more recent work, NML has been adapted to the conditional setting to produce models that are much better calibrated and maintain a notion of uncertainty, while achieving optimal worst case classification regret. Given the challenges of overconfidence described above, this is an ideal choice for the problem of reward inference.

Rather than simply training models via maximum likelihood, CNML performs a more complex inference procedure to produce likelihoods for any point that is being queried for its label. Intuitively, CNML constructs a set of different maximum likelihood problems by labeling a particular query point with every possible label value that it might take, then outputs a final prediction based on how easily it was able to adapt to each of those proposed labels given the entire dataset observed thus far. Given a particular query point , and a prior dataset , CNML solves k different maximum likelihood problems and normalizes them to produce the desired label likelihood , where represents the number of possible values that the label may take. Formally, given a model , loss function , training dataset with classes , and a new query point , CNML solves the following maximum likelihood problems:

It then generates predictions for each of the classes using their corresponding models, and normalizes the results for its final output:

Comparison of outputs from a standard classifier and a CNML classifier. CNML outputs more conservative predictions on points that are far from the training distribution, indicating uncertainty about those points’ true outputs. (Credit: Aurick Zhou, BAIR Blog)

Intuitively, if the query point is farther from the original training distribution represented by D, CNML will be able to more easily adapt to any arbitrary label in , making the resulting predictions closer to uniform. In this way, CNML is able to produce better calibrated predictions, and maintain a clear notion of uncertainty based on which data point is being queried.

Leveraging CNML-based classifiers for Reward Inference

Given the above background on CNML as a means to produce better calibrated classifiers, it becomes clear that this provides us a straightforward way to address the overconfidence problem with classifier based rewards in outcome driven RL. By replacing a standard maximum likelihood classifier with one trained using CNML, we are able to capture a notion of uncertainty and obtain directed exploration for outcome driven RL. In fact, in the discrete case, CNML corresponds to imposing a uniform prior on the output space — in an RL setting, this is equivalent to using a count-based exploration bonus as the reward function. This turns out to give us a very appropriate notion of uncertainty in the rewards, and solves many of the exploration challenges present in classifier based RL.

However, we don’t usually operate in the discrete case. In most cases, we use expressive function approximators and the resulting representations of different states in the world share similarities. When a CNML based classifier is learned in this scenario, with expressive function approximation, we see that it can provide more than just task agnostic exploration. In fact, it can provide a directed notion of reward shaping, which guides an agent towards the goal rather than simply encouraging it to expand the visited region naively. As visualized below, CNML encourages exploration by giving optimistic success probabilities in less-visited regions, while also providing better shaping towards the goal.

As we will show in our experimental results, this intuition scales to higher dimensional problems and more complex state and action spaces, enabling CNML based rewards to solve significantly more challenging tasks than is possible with typical classifier based rewards.

However, on closer inspection of the CNML procedure, a major challenge becomes apparent. Each time a query is made to the CNML classifier, different maximum likelihood problems need to be solved to convergence, then normalized to produce the desired likelihood. As the size of the dataset increases, as it naturally does in reinforcement learning, this becomes a prohibitively slow process. In fact, as seen in Table 1, RL with standard CNML based rewards takes around 4 hours to train a single epoch (1000 timesteps). Following this procedure blindly would take over a month to train a single RL agent, necessitating a more time efficient solution. This is where we find meta-learning to be a crucial tool.

Meta-Learning CNML Classifiers Meta-learning is a tool that has seen a lot of use cases in few-shot learning for image classification, learning quicker optimizers and even learning more efficient RL algorithms. In essence, the idea behind meta-learning is to leverage a set of “meta-training” tasks to learn a model (and often an adaptation procedure) that can very quickly adapt to a new task drawn from the same distribution of problems.

Meta-learning techniques are particularly well suited to our class of computational problems since it involves quickly solving multiple different maximum likelihood problems to evaluate the CNML likelihood. Each the maximum likelihood problems share significant similarities with each other, enabling a meta-learning algorithm to very quickly adapt to produce solutions for each individual problem. In doing so, meta-learning provides us an effective tool for producing estimates of normalized maximum likelihood significantly more quickly than possible before.

The intuition behind how to apply meta-learning to the CNML (meta-NML) can be understood by the graphic above. For a data-set of points, meta-NML would first construct tasks, corresponding to the positive and negative maximum likelihood problems for each datapoint in the dataset. Given these constructed tasks as a (meta) training set, a meta–learning algorithm can be applied to learn a model that can very quickly be adapted to produce solutions to any of these maximum likelihood problems. Equipped with this scheme to very quickly solve maximum likelihood problems, producing CNML predictions around x faster than possible before. Prior work studied this problem from a Bayesian approach, but we found that it often scales poorly for the problems we considered.

Equipped with a tool for efficiently producing predictions from the CNML distribution, we can now return to the goal of solving outcome-driven RL with uncertainty aware classifiers, resulting in an algorithm we call MURAL.

MURAL: Meta-Learning Uncertainty-Aware Rewards for Automated Reinforcement Learning To more effectively solve outcome driven RL problems, we incorporate meta-NML into the standard classifier based procedure as follows: After each epoch of RL, we sample a batch of points from the replay buffer and use them to construct meta-tasks. We then run iteration of meta-training on our model.

We assign rewards using NML, where the NML outputs are approximated using only one gradient step for each input point.

The resulting algorithm, which we call MURAL, replaces the classifier portion of standard classifier-based RL algorithms with a meta-NML model instead. Although meta-NML can only evaluate input points one at a time instead of in batches, it is substantially faster than naive CNML, and MURAL is still comparable in runtime to standard classifier-based RL, as shown in Table 1 below.

Table 1. Runtimes for a single epoch of RL on the 2D maze task.

We evaluate MURAL on a variety of navigation and robotic manipulation tasks, which present several challenges including local optima and difficult exploration. MURAL solves all of these tasks successfully, outperforming prior classifier-based methods as well as standard RL with exploration bonuses.

Visualization of behaviors learned by MURAL. MURAL is able to perform a variety of behaviors in navigation and manipulation tasks, inferring rewards from outcome examples.

Quantitative comparison of MURAL to baselines. MURAL is able to outperform baselines which perform task-agnostic exploration, standard maximum likelihood classifiers.

This suggests that using meta-NML based classifiers for outcome driven RL provides us an effective way to provide rewards for RL problems, providing benefits both in terms of exploration and directed reward shaping.

Takeaways In conclusion, we showed how outcome driven RL can define a class of more tractable RL problems. Standard methods using classifiers can often fall short in these settings as they are unable to provide any benefits of exploration or guidance towards the goal. Leveraging a scheme for training uncertainty aware classifiers via conditional normalized maximum likelihood allows us to more effectively solve this problem, providing benefits in terms of exploration and reward shaping towards successful outcomes. The general principles defined in this work suggest that considering tractable approximations to the general RL problem may allow us to simplify the challenge of reward specification and exploration in RL while still encompassing a rich class of control problems.

This post is based on the paper “MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning”, which was presented at ICML 2021. You can see results on our website, and we provide code to reproduce our experiments.

View Details

The IEEE International Conference on Automation Science and Engineering (CASE) is the flagship automation conference of the IEEE Robotics and Automation Society and constitutes the primary forum for cross-industry and multidisciplinary research in automation. Its goal is to provide a broad coverage and dissemination of foundational research in automation among researchers, academics, and practitioners. Here we bring you the online presentations by the finalists of the four awards given at the conference. Congratulations to all the finalists and winners!

Best student paper award Winner * Designing a User-Centred and Data-Driven Controller for Pushrim-Activated Power-Assisted Wheels: A Case Study

Mahsa Khalili, H.F. Machiel Van der Loos and Jaimie Borisoff

Finalists * Including Sparse Production Knowledge into Variational Autoencoders to Increase Anomaly Detection Reliability

Tom Hammerbacher, Markus Lange-Hegermann, Gorden Platz

• Synthesis and Implementation of Distributed Supervisory Controllers with Communication Delays

Lars Moormann, Reinier Hendrik Jacob Schouten, Joanna Maria Van de Mortel-Fronczak, Wan Fokkink, Jacobus E. Rooda

• Optimal Planning of Internet Data Centers Decarbonized by Hydrogen-Water-Based Energy Systems

Jinhui Liu, Zhanbo Xu, Jiang Wu, kun liu, Xunhang Sun, Xiaohong Guan

• Deep Reinforcement Learning for Prefab Assembly Planning in Robot-Based Prefabricated Construction

Zhu Aiyu, Gangyan Xu, Pieter Pauwels, Bauke de Vries, Meng Fang

• Singularity-Aware Motion Planning for Multi-Axis Additive Manufacturing

Charlie C.L. Wang, Tianyu Zhang, Xiangjia Chen, Guoxin Fang, Yingjun Tian

Best conference paper award Winner * Extended Fabrication-Aware Convolution Learning Framework for Predicting 3D Shape Deformation in Additive Manufacturing

Yuanxiang Wang, Cesar Ruiz, Qiang Huang

Finalists * Probabilistic Movement Primitive Control Via Control Barrier Functions

Mohammadreza Davoodi, Asif Iqbal, Joe Cloud, William Beksi, Nicholas Gans

• Efficient Optimization-Based Falsification of Cyber-Physical Systems with Multiple Conjunctive Requirements

Logan Mathesen, Giulia Pedrielli, Georgios Fainekos

Best application paper award Winner * A Seamless Workflow for Design and Fabrication of Multimaterial Pneumatic Soft Actuators

Lawrence Smith, Travis Hainsworth, Zachary Jordan, Xavier Bell, Robert MacCurdy

Finalists * Dynamic Multi-Goal Motion Planning with Range Constraints for Autonomous Underwater Vehicles Following Surface Vehicles

James McMahon, Erion Plaku

• OpenUAV Cloud Testbed: a Collaborative Design Studio for Field Robotics

Harish Anand, Stephen A. Rees, Zhiang Chen, Ashwin Jose Poruthukaran, Sarah Bearman, Lakshmi Gana Prasad Antervedi, Jnaneshwar Das

Best healthcare automation paper award Winner * Hospital Beds Planning and Admission Control Policies for COVID-19 Pandemic: A Hybrid Computer Simulation Approach

Yiruo Lu, Yongpei Guan, Xiang Zhong, Jennifer Fishe, Thanh Hogan

Finalists * Rollout-Based Gantry Call-Back Control for Proton Therapy Systems

Feifan Wang, Yu-Li Huang, Feng Ju

• Progress in Development of an Automated Mosquito Salivary Gland Extractor: A Step Forward to Malaria Vaccine Mass Production

Wanze Li, Zhuoqun Zhang, Zhuohong He, Parth Vora, Alan Lai, Balazs Vagvolgyi, Simon Leonard, Anna Goodridge, Ioan Iulian Iordachita, Stephen L. Hoffman, Sumana Chakravarty, B Kim Lee Sim, Russell H. Taylor

View Details

Amit Goel, Director of Product Management for Autonomous Machines at NVIDIA, discusses the new collaboration between Open Robotics and NVIDIA. The collaboration will improve the way ROS and NVIDIA’s line of products such as Isaac SIM and the Jetson line of embedded boards operate together.

NVIDIA’s Isaac SIM lets developers build robust and scalable simulations. Dramatically reducing the costs of capturing real-world data and speeding up development time.

Their Jetson line of embedded boards is core to many robotics architectures, leveraging hardware-optimized chips for machine learning, computer vision, video processing, and more.

The improvements to ROS will allow robotics companies to better utilize the available computational power, while still developing on the robotics-centric platform familiar to many.

For the developers out there, check out the ISAAC ROS Github repository to implement this yourself.

Amit Goel

Amit Goel is Director of Product Management for Autonomous Machines at NVIDIA, where he leads the product development of NVIDIA Jetson, the most advanced platform for AI computing at the edge.

Amit has more than 15 years of experience in the technology industry working in both software and hardware design roles. Prior to joining NVIDIA in 2011, he worked as a senior software engineer at Synopsys, where he developed algorithms for statistical performance modeling of digital designs.

Amit holds a Bachelor of Engineering in electronics and communication from Delhi College of Engineering, a Master of Science in electrical engineering from Arizona State University, and an MBA from the University of California at Berkeley.

Links

• Download mp3 (29.8 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

View Details

MIT researchers have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. Credits: Photo courtesy of the researchers

By Adam Zewe | MIT News Office

A loping cheetah dashes across a rolling field, bounding over sudden gaps in the rugged terrain. The movement may look effortless, but getting a robot to move this way is an altogether different prospect.

In recent years, four-legged robots inspired by the movement of cheetahs and other animals have made great leaps forward, yet they still lag behind their mammalian counterparts when it comes to traveling across a landscape with rapid elevation changes.

“In those settings, you need to use vision in order to avoid failure. For example, stepping in a gap is difficult to avoid if you can’t see it. Although there are some existing methods for incorporating vision into legged locomotion, most of them aren’t really suitable for use with emerging agile robotic systems,” says Gabriel Margolis, a PhD student in the lab of Pulkit Agrawal, professor in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT.

Now, Margolis and his collaborators have developed a system that improves the speed and agility of legged robots as they jump across gaps in the terrain. The novel control system is split into two parts — one that processes real-time input from a video camera mounted on the front of the robot and another that translates that information into instructions for how the robot should move its body. The researchers tested their system on the MIT mini cheetah, a powerful, agile robot built in the lab of Sangbae Kim, professor of mechanical engineering.

Unlike other methods for controlling a four-legged robot, this two-part system does not require the terrain to be mapped in advance, so the robot can go anywhere. In the future, this could enable robots to charge off into the woods on an emergency response mission or climb a flight of stairs to deliver medication to an elderly shut-in.

Margolis wrote the paper with senior author Pulkit Agrawal, who heads the Improbable AI lab at MIT and is the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; Professor Sangbae Kim in the Department of Mechanical Engineering at MIT; and fellow graduate students Tao Chen and Xiang Fu at MIT. Other co-authors include Kartik Paigwar, a graduate student at Arizona State University; and Donghyun Kim, an assistant professor at the University of Massachusetts at Amherst. The work will be presented next month at the Conference on Robot Learning.

It’s all under control The use of two separate controllers working together makes this system especially innovative.

A controller is an algorithm that will convert the robot’s state into a set of actions for it to follow. Many blind controllers — those that do not incorporate vision — are robust and effective but only enable robots to walk over continuous terrain.

Vision is such a complex sensory input to process that these algorithms are unable to handle it efficiently. Systems that do incorporate vision usually rely on a “heightmap” of the terrain, which must be either preconstructed or generated on the fly, a process that is typically slow and prone to failure if the heightmap is incorrect.

To develop their system, the researchers took the best elements from these robust, blind controllers and combined them with a separate module that handles vision in real-time.

The robot’s camera captures depth images of the upcoming terrain, which are fed to a high-level controller along with information about the state of the robot’s body (joint angles, body orientation, etc.). The high-level controller is a neural network that “learns” from experience.

That neural network outputs a target trajectory, which the second controller uses to come up with torques for each of the robot’s 12 joints. This low-level controller is not a neural network and instead relies on a set of concise, physical equations that describe the robot’s motion.

“The hierarchy, including the use of this low-level controller, enables us to constrain the robot’s behavior so it is more well-behaved. With this low-level controller, we are using well-specified models that we can impose constraints on, which isn’t usually possible in a learning-based network,” Margolis says.

Teaching the network The researchers used the trial-and-error method known as reinforcement learning to train the high-level controller. They conducted simulations of the robot running across hundreds of different discontinuous terrainsand rewarded it for successful crossings.

Over time, the algorithm learned which actions maximized the reward.

Then they built a physical, gapped terrain with a set of wooden planks and put their control scheme to the test using the mini cheetah.

“It was definitely fun to work with a robot that was designed in-house at MIT by some of our collaborators. The mini cheetah is a great platform because it is modular and made mostly from parts that you can order online, so if we wanted a new battery or camera, it was just a simple matter of ordering it from a regular supplier and, with a little bit of help from Sangbae’s lab, installing it,” Margolis says.

From left to right: PhD students Tao Chen and Gabriel Margolis; Pulkit Agrawal, the Steven G. and Renee Finn Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science; and PhD student Xiang Fu. Credits: Photo courtesy of the researchers

Estimating the robot’s state proved to be a challenge in some cases. Unlike in simulation, real-world sensors encounter noise that can accumulate and affect the outcome. So, for some experiments that involved high-precision foot placement, the researchers used a motion capture system to measure the robot’s true position.

Their system outperformed others that only use one controller, and the mini cheetah successfully crossed 90 percent of the terrains.

“One novelty of our system is that it does adjust the robot’s gait. If a human were trying to leap across a really wide gap, they might start by running really fast to build up speed and then they might put both feet together to have a really powerful leap across the gap. In the same way, our robot can adjust the timings and duration of its foot contacts to better traverse the terrain,” Margolis says.

Leaping out of the lab While the researchers were able to demonstrate that their control scheme works in a laboratory, they still have a long way to go before they can deploy the system in the real world, Margolis says.

In the future, they hope to mount a more powerful computer to the robot so it can do all its computation on board. They also want to improve the robot’s state estimator to eliminate the need for the motion capture system. In addition, they’d like to improve the low-level controller so it can exploit the robot’s full range of motion, and enhance the high-level controller so it works well in different lighting conditions.

“It is remarkable to witness the flexibility of machine learning techniques capable of bypassing carefully designed intermediate processes (e.g. state estimation and trajectory planning) that centuries-old model-based techniques have relied on,” Kim says. “I am excited about the future of mobile robots with more robust vision processing trained specifically for locomotion.”

The research is supported, in part, by the MIT’s Improbable AI Lab, Biomimetic Robotics Laboratory, NAVER LABS, and the DARPA Machine Common Sense Program.

View Details

Robotics Today held three more online talks since we published the one from Amanda Prorok (Learning to Communicate in Multi-Agent Systems). In this post we bring you the last talks that Robotics Today (currently on hiatus) uploaded to their YouTube channel: Raia Hadsell from DeepMind talking about ‘Scalable Robot Learning in Rich Environments’, Koushil Sreenath from UC Berkeley talking about ‘Safety-Critical Control for Dynamic Robots’, and Antonio Bicchi from the Istituto Italiano di Tecnologia talking about ‘Planning and Learning Interaction with Variable Impedance’.

| | Raia Hadsell (DeepMind) – Scalable Robot Learning in Rich Environments Abstract: As modern machine learning methods push towards breakthroughs in controlling physical systems, games and simple physical simulations are often used as the main benchmark domains. As the field matures, it is important to develop more sophisticated learning systems with the aim of solving more complex real-world tasks, but problems like catastrophic forgetting and data efficiency remain critical, particularly for robotic domains. This talk will cover some of the challenges that exist for learning from interactions in more complex, constrained, and real-world settings, and some promising new approaches that have emerged. Bio: Raia Hadsell is the Director of Robotics at DeepMind. Dr. Hadsell joined DeepMind in 2014 to pursue new solutions for artificial general intelligence. Her research focuses on the challenge of continual learning for AI agents and robots, and she has proposed neural approaches such as policy distillation, progressive nets, and elastic weight consolidation to solve the problem of catastrophic forgetting. Dr. Hadsell is on the executive boards of ICLR (International Conference on Learning Representations), WiML (Women in Machine Learning), and CoRL (Conference on Robot Learning). She is a fellow of the European Lab on Learning Systems (ELLIS), a founding organizer of NAISys (Neuroscience for AI Systems), and serves as a CIFAR advisor. |

| | Koushil Sreenath (UC Berkeley) – Safety-Critical Control for Dynamic Robots: A Model-based and Data-driven Approach Abstract: Model-based controllers can be designed to provide guarantees on stability and safety for dynamical systems. In this talk, I will show how we can address the challenges of stability through control Lyapunov functions (CLFs), input and state constraints through CLF-based quadratic programs, and safety-critical constraints through control barrier functions (CBFs). However, the performance of model-based controllers is dependent on having a precise model of the system. Model uncertainty could lead not only to poor performance but could also destabilize the system as well as violate safety constraints. I will present recent results on using model-based control along with data-driven methods to address stability and safety for systems with uncertain dynamics. In particular, I will show how reinforcement learning as well as Gaussian process regression can be used along with CLF and CBF-based control to address the adverse effects of model uncertainty. Bio: Koushil Sreenath is an Assistant Professor of Mechanical Engineering, at UC Berkeley. He received a Ph.D. degree in Electrical Engineering and Computer Science and a M.S. degree in Applied Mathematics from the University of Michigan at Ann Arbor, MI, in 2011. He was a Postdoctoral Scholar at the GRASP Lab at University of Pennsylvania from 2011 to 2013 and an Assistant Professor at Carnegie Mellon University from 2013 to 2017. His research interest lies at the intersection of highly dynamic robotics and applied nonlinear control. His work on dynamic legged locomotion was featured on The Discovery Channel, CNN, ESPN, FOX, and CBS. His work on dynamic aerial manipulation was featured on the IEEE Spectrum, New Scientist, and Huffington Post. His work on adaptive sampling with mobile sensor networks was published as a book. He received the NSF CAREER, Hellman Fellow, Best Paper Award at the Robotics: Science and Systems (RSS), and the Google Faculty Research Award in Robotics. |

| | Antonio Bicchi (Istituto Italiano di Tecnologia) – Planning and Learning Interaction with Variable Impedance Abstract: In animals and in humans, the mechanical impedance of their limbs changes not only in dependence of the task, but also during different phases of the execution of a task. Part of this variability is intentionally controlled, by either co-activating muscles or by changing the arm posture, or both. In robots, impedance can be varied by varying controller gains, stiffness of hardware parts, and arm postures. The choice of impedance profiles to be applied can be planned off-line, or varied in real time based on feedback from the environmental interaction. Planning and control of variable impedance can use insight from human observations, from mathematical optimization methods, or from learning. In this talk I will review the basics of human and robot variable impedance, and discuss how this impact applications ranging from industrial and service robotics to prosthetics and rehabilitation. Bio: Antonio Bicchi is a scientist interested in robotics and intelligent machines. After graduating in Pisa and receiving a Ph.D. from the University of Bologna, he spent a few years at the MIT AI Lab of Cambridge before becoming Professor in Robotics at the University of Pisa. In 2009 he founded the Soft Robotics Laboratory at the Italian Institute of Technology in Genoa. Since 2013 he is Adjunct Professor at Arizona State University, Tempe, AZ. He has coordinated many international projects, including four grants from the European Research Council (ERC). He served the research community in several ways, including by launching the WorldHaptics conference and the IEEE Robotics and Automation Letters. He is currently the President of the Italian Institute of Robotics and Intelligent Machines. He has authored over 500 scientific papers cited more than 25,000 times. He supervised over 60 doctoral students and more than 20 postdocs, most of whom are now professors in universities and international research centers, or have launched their own spin-off companies. His students have received prestigious awards, including three first prizes and two nominations for the best theses in Europe on robotics and haptics. He is a Fellow of IEEE since 2005. In 2018 he received the prestigious IEEE Saridis Leadership Award. |

View Details

In this episode, Audrow Nash interviews Erik Schluntz, co-founder and CTO of Cobalt Robotics, which makes a security guard robot. Erik speaks about how their robot handles elevators, how they have humans-in-the-loop to help their robot make decisions, robot body language, and gives advice for entrepreneurs.

Episode Links

• Cobalt’s website
• Erik’s website
• Video introducing the Cobalt robot
• Video of the Cobalt robot using an elevator

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes playlist
• Clips playlist

View Details

Researchers at MIT have developed a fully-integrated robotic arm that fuses visual data from a camera and radio frequency (RF) information from an antenna to find and retrieve objects, even when they are buried under a pile and fully out of view. Credits: Courtesy of the researchers

By Adam Zewe | MIT News Office

A busy commuter is ready to walk out the door, only to realize they’ve misplaced their keys and must search through piles of stuff to find them. Rapidly sifting through clutter, they wish they could figure out which pile was hiding the keys.

Researchers at MIT have created a robotic system that can do just that. The system, RFusion, is a robotic arm with a camera and radio frequency (RF) antenna attached to its gripper. It fuses signals from the antenna with visual input from the camera to locate and retrieve an item, even if the item is buried under a pile and completely out of view.

The RFusion prototype the researchers developed relies on RFID tags, which are cheap, battery-less tags that can be stuck to an item and reflect signals sent by an antenna. Because RF signals can travel through most surfaces (like the mound of dirty laundry that may be obscuring the keys), RFusion is able to locate a tagged item within a pile.

Using machine learning, the robotic arm automatically zeroes-in on the object’s exact location, moves the items on top of it, grasps the object, and verifies that it picked up the right thing. The camera, antenna, robotic arm, and AI are fully integrated, so RFusion can work in any environment without requiring a special set up.

In this video still, the robotic arm is looking for keys hidden underneath items. Credits: Courtesy of the researchers

While finding lost keys is helpful, RFusion could have many broader applications in the future, like sorting through piles to fulfill orders in a warehouse, identifying and installing components in an auto manufacturing plant, or helping an elderly individual perform daily tasks in the home, though the current prototype isn’t quite fast enough yet for these uses.

“This idea of being able to find items in a chaotic world is an open problem that we’ve been working on for a few years. Having robots that are able to search for things under a pile is a growing need in industry today. Right now, you can think of this as a Roomba on steroids, but in the near term, this could have a lot of applications in manufacturing and warehouse environments,” said senior author Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab.

Co-authors include research assistant Tara Boroushaki, the lead author; electrical engineering and computer science graduate student Isaac Perper; research associate Mergen Nachin; and Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering. The research will be presented at the Association for Computing Machinery Conference on Embedded Networked Senor Systems next month.

Sending signals RFusion begins searching for an object using its antenna, which bounces signals off the RFID tag (like sunlight being reflected off a mirror) to identify a spherical area in which the tag is located. It combines that sphere with the camera input, which narrows down the object’s location. For instance, the item can’t be located on an area of a table that is empty.

But once the robot has a general idea of where the item is, it would need to swing its arm widely around the room taking additional measurements to come up with the exact location, which is slow and inefficient.

The researchers used reinforcement learning to train a neural network that can optimize the robot’s trajectory to the object. In reinforcement learning, the algorithm is trained through trial and error with a reward system.

“This is also how our brain learns. We get rewarded from our teachers, from our parents, from a computer game, etc. The same thing happens in reinforcement learning. We let the agent make mistakes or do something right and then we punish or reward the network. This is how the network learns something that is really hard for it to model,” Boroushaki explains.

In the case of RFusion, the optimization algorithm was rewarded when it limited the number of moves it had to make to localize the item and the distance it had to travel to pick it up.

Once the system identifies the exact right spot, the neural network uses combined RF and visual information to predict how the robotic arm should grasp the object, including the angle of the hand and the width of the gripper, and whether it must remove other items first. It also scans the item’s tag one last time to make sure it picked up the right object.

Cutting through clutter The researchers tested RFusion in several different environments. They buried a keychain in a box full of clutter and hid a remote control under a pile of items on a couch.

But if they fed all the camera data and RF measurements to the reinforcement learning algorithm, it would have overwhelmed the system. So, drawing on the method a GPS uses to consolidate data from satellites, they summarized the RF measurements and limited the visual data to the area right in front of the robot.

Their approach worked well — RFusion had a 96 percent success rate when retrieving objects that were fully hidden under a pile.

“We let the agent make mistakes or do something right and then we punish or reward the network. This is how the network learns something that is really hard for it to model,” co-author Tara Boroushaki, pictured here, explains. Credits: Courtesy of the researchers

“Sometimes, if you only rely on RF measurements, there is going to be an outlier, and if you rely only on vision, there is sometimes going to be a mistake from the camera. But if you combine them, they are going to correct each other. That is what made the system so robust,” Boroushaki says.

In the future, the researchers hope to increase the speed of the system so it can move smoothly, rather than stopping periodically to take measurements. This would enable RFusion to be deployed in a fast-paced manufacturing or warehouse setting.

Beyond its potential industrial uses, a system like this could even be incorporated into future smart homes to assist people with any number of household tasks, Boroushaki says.

“Every year, billions of RFID tags are used to identify objects in today’s complex supply chains, including clothing and lots of other consumer goods. The RFusion approach points the way to autonomous robots that can dig through a pile of mixed items and sort them out using the data stored in the RFID tags, much more efficiently than having to inspect each item individually, especially when the items look similar to a computer vision system,” says Matthew S. Reynolds, CoMotion Presidential Innovation Fellow and associate professor of electrical and computer engineering at the University of Washington, who was not involved in the research. “The RFusion approach is a great step forward for robotics operating in complex supply chains where identifying and ‘picking’ the right item quickly and accurately is the key to getting orders fulfilled on time and keeping demanding customers happy.”

The research is sponsored by the National Science Foundation, a Sloan Research Fellowship, NTT DATA, Toppan, Toppan Forms, and the Abdul Latif Jameel Water and Food Systems Lab.

View Details

If you visited Robohub this week, you may have spotted a big change: how this blog looks now! On Tuesday (coinciding with Ada Lovelace Day and our ‘50 women in robotics that you need to know about‘ by chance), Robohub got a massive modernisation on its look by our technical director Ioannis K. Erripis and his team.

There are many improvements and new features but the biggest update apart from the code is the design which is more clean and simple looking (especially on the single post view).

Ioannis K. Erripis

This fresher look has recently been tested on our sister project, AIhub.org. As Ioannis says, it offers a cleaner, simpler and more readable way to access the content from the robotics community that we post in this blog. You will also notice that the main page now displays a mosaic of content which is more accessible than before to facilitate how you scroll down our site.

Your feedback matters to us: if you find any issue with the new design or have any comment/idea for improvement, please let us know! You can reach us at editors[at]robohub.org.

And there are even more exciting news coming up together with this fresh look! We will soon release a brand new project that we have been developing in the background during this year. Stay tuned!

View Details

Robohub Podcast · Public Transit

In this episode, our interviewer Lilly speaks to Alex Wallar, co-founder and CTO of The Routing Company. Wallar shares his background in multi-robot path-planning and optimization, and his research on scheduling and routing algorithms for high-capacity ride-sharing. They discuss how The Routing Company helps cities meet the needs of their people, the technical ins and outs of their dispatcher and assignment system, and the importance of public transit to cities and their economics.

Alex Wallar

Alex Wallar is Co-founder and CTO of the Routing Company, an on-demand vehicle routing and management platform that partners with cities to power the future of public transit. Previously he was pursuing a PhD in the Computer Science and Artificial Intelligence Lab at MIT, conducting research on mathematical optimization for high-capacity ride-sharing.

Links

• Download mp3 (41.7 MB)
• Subscribe to Robohub using iTunes, RSS, or Spotify
• Support us on Patreon

View Details

It’s Ada Lovelace Day and once again we’re delighted to introduce you to “50 women in robotics you need to know about”! From the Afghanistan Girls Robotics Team to K.G.Engelhardt who in 1989 founded, and was the first Director of, the Center for Human Service Robotics at Carnegie Mellon, these women showcase a wide range of roles in robotics. We hope these short bios will provide a world of inspiration, in our ninth Women in Robotics list! Tweet this.

In 2021, we showcase women in robotics in Afghanistan, Australia, Canada, Denmark, Finland, France, Germany, Hong Kong, India, Iran, Ireland, Israel, Italy, Japan, New Zealand, Portugal, Singapore, South Africa, Spain, Switzerland, Russia, United Kingdom and United States. They are researchers, industry leaders, and artists. Some women are at the start of their careers, while others have literally written the book, the program or the standards.

It is, however, disturbing how hard it can be to find records of women who were an important part of the history of robotics, such as K.G. Engelhardt. Statistically speaking, women are far more likely to leave the workforce or change careers due to family pressures, and that contributes to the erasure. Last year, we talked about the importance of having more equitable citation counts. The citation problem is expected to significantly disadvantage women and people of color due to the historical lack of women followed by the recent growth of large scientific teams, multiplying exclusion.

A more dangerous form of erasure is happening today in Afghanistan. The Afghanistan Girls Robotics Team was forced to flee the country, thanks to help from their support group, the Digital Citizen Fund. What steps must the international community take in support of a future for Afghan girls’ education? Hear from UN Deputy Secretary-General Amina Mohammed, Nobel Laureate and Malala Fund Co-Founder Malala Yousafzai, and Somaya Faruqi, Captain of the Afghan Dreamers Robotics Team, in this recent UN video.

You can support The Afghan Dreamers here, and soon watch a documentary about the original team which was shot with the girls in Afghanistan in 2019 and 2020.

Meanwhile, UC Davis developed an online digital backpack to keep academic credentials and school records safe and private. The Article 26 Backpack references the 1948 Universal Declaration of Human Rights, and the right to an education. At the moment, the Backpack is for people 18 and over, with a high school diploma or baccalaureate, whose education has been affected by war, conflict or economic conditions.

On the good news front, the IEEE RAS Women in Engineering (WIE) Committee recently completed a several year study of gender representation in conference leading roles at RAS-supported conferences. Individuals who hold these roles select organizing committees, choose speakers, and make final decisions on paper acceptances. In this video, the authors lead a discussion about the findings and the story behind the study. In addition to presenting detailed data and releasing anonymized datasets for further study, the authors provided suggestions on changes to help ensure a more diverse and representative robotics community where anyone can thrive. The paper “Gender Diversity of Conference Leadership” by Laura Graesser, Aleksandra Faust, Hadas Kress-Gazit, Lydia Tapia, and Risa Ulinskiby was in the June 2021 IEEE Robotics and Automation Magazine, with a follow up “Retrospective on a Watershed Moment for IEEE Robotics and Automation Society Gender Diversity [Women in Engineering]” in Sep 2021 by Lydia Tapia reporting on gender diversity initiatives undertaken by the Robotics and Automation Society.

We publish this list because the lack of visibility of women in robotics leads to the unconscious perception that women aren’t making newsworthy contributions. We encourage you to use our lists to help find women for keynotes, panels, interviews and to cite their research and include them in curricula. Tulane University published a guide to help you calculate how much of your reading list includes female authors and a citation guide, similar to the CiteHer campaign from BlackComputeher.org.

And Women in Robotics have just launched a Photo Challenge! We got so tired of seeing literally hundreds of images of female robots showing up whenever we searched for images of women building robots, or images of men building robots while women watched. Let’s push Sophia out of the top search results and showcase real women building real robots instead!

We hope you are inspired by these profiles, and if you want to work in robotics too, please join us at Women in Robotics. We are now a 501(c)(3) non-profit organization, but even so, this post wouldn’t be possible if not for the hard work of volunteers; Andra Keay, Fatemeh Pahlevan Aghababa, Jeana diNatale and Daniel Carrillo Zapata. Tweet this

And we encourage #womeninrobotics and women who’d like to work in robotics to join our professional network at http://womeninrobotics.org Want to keep reading? There are more than 200 other stories on our 2013 to 2020 lists (and their updates):

• 30 women in robotics you need to know about (2020)
• 30 women in robotics you need to know about (2019)
• 25 women in robotics you need to know about (2018)
• 25 women in robotics you need to know about (2017)
• 25 women in robotics you need to know about (2016)
• 25 women in robotics you need to know about (2015)
• 25 women in robotics you need to know about (2014)
• 25 women in robotics you need to know about (2013)

Why not nominate someone for inclusion next year! Tweet this.

View Details

How can women feel as if they belong in robotics if we can’t see any pictures of women building or programming robots? The Civil Rights Activist Marian Wright Edelson aptly said, “You can’t be what you can’t see.” We’d like you all to take photos of women building and coding robots and share them with us!

Here’s the handy guide to what a great photo looks like with some awesome examples. This is a great opportunity for research labs and robotics companies to showcase their talented women and other underrepresented groups.

View Details

In this episode, Audrow Nash speaks with Melonee Wise, former CEO of Fetch Robotics and current VP of Robotics Automation at Zebra Technologies. Melonee speaks about the origin of Fetch Robotics, her experience at Willow Garage, her experience being acquired by Zebra Technologies, challenges in the warehouse setting, on autonomous cars, and on the future of robotics.

Links * Download the episode * Melonee Wise’s website * Fetch Robotics * Zebra Technologies

Subscribe

• YouTube
• Apple Podcasts
• Spotify
• Google Podcasts
• Overcast
• Pocket Casts

View Details

On the 12th of October, the world will celebrate Ada Lovelace Day to honor the achievements of women in science, technology, engineering and maths (STEM). After a successful worldwide online celebration of Ada Lovelace Day last year, this year’s celebration returns with a stronger commitment to online inclusion. In Finding Ada (the main network supporting Ada Lovelace Day), there will be three free webinars that you can enjoy in the comfort of your own home. There will also be loads of events happening around the world, so you have a wide range of content to celebrate Ada Lovelace Day 2021!

Engineering – Solving Problems for Real People

Engineering is the science of problem solving, and we have some pretty big problems in front of us. So how are engineers tackling the COVID-19 pandemic and climate change? And how do they stay focused on the impact of their engineering solutions on people and communities?

In partnership with STEM Wana Trust, we invite you to join Renée Young, associate mechanical engineer at Beca, Victoria Clark, senior environmental engineer at Beca, Natasha Mudaliar, operations manager at Reliance Reinforcing, and Sujata Roy, system planning engineer at Transpower, for a fascinating conversation about the challenges and opportunities of engineering.

13:00 NZST, 12 Oct: Perfect for people in New Zealand, Australia, and the Americas. (Note for American audiences: This panel will be on Monday for you.)

Register here, and find out about the speakers here.

Fusing Tech & Art in Games

The Technical Artist is a new kind of role in the games industry, but the possibilities for those who create and merge art and technology is endless. So what is tech art? And how are tech artists pushing the boundaries and creating new experiences for players?

Ada Lovelace Day and Ukie’s #RaiseTheGame invite you to join tech artist Kristrun Fridriksdottir, Jodie Azhar, technical art director at Silver Rain Games, Emma Roseburgh from Avalanche Studios, and Laurène Hurlin from Pixel Toys for our tech art webinar.

13:00 BST, 12 Oct: Perfect for people in the UK, Europe, Africa, Middle East, India, for early birds in the Americas and night owls in AsiaPacific.

Register here, and find out about the speakers here.

The Science of Hypersleep

Hypersleep is a common theme in science fiction, but what does science have to say about putting humans into suspended animation? What can we learn from hibernating animals? What’s the difference between hibernation and sleep? What health impacts would extended hypersleep have?

Ada Lovelace Day and the Arthur C. Clarke Award invite you to join science fiction author Anne Charnock, Prof Gina Poe, an expert on the relationship between sleep and memory, Dr Anusha Shankar, who studies torpor in hummingbirds, and Prof Kelly Drew, who studies hibernation in squirrels, for a discussion of whether hypersleep in humans is possible.

19:00 BST, 12 Oct: Perfect for people in the UK, Europe, Africa, and the Americas.

Register here, and find out about the speakers here.

Other worldwide events Apart from the three webinars above, many other organisations will hold their own events to celebrate the day. From a 24-hour global edit-a-thon (The Pankhurst Centre) to a digital theatre play (STEM on Stage) to an online machine learning breakfast (Square Women Engineers + Allies Australia), plus several talks and panel discussions like this one on how you can change the world with the help of physics (Founders4Schools), or this other one on inspiring women and girls in STEAM (Engine Shed), you have plenty of options to choose from.

For a full overview of international events, check out this website.

We also hope that you enjoy reading our annual list of women in robotics that you need to know that will be released on the day. Happy Ada Lovelace Day 2021!

View Details

When it comes to exploring complex and unknown environments such as forests, buildings or caves, drones are hard to beat. They are fast, agile and small, and they can carry sensors and payloads virtually everywhere. However, autonomous drones can hardly find their way through an unknown environment without a map. For the moment, expert human pilots are needed to release the full potential of drones.

“To master autonomous agile flight, you need to understand the environment in a split second to fly the drone along collision-free paths,” says Davide Scaramuzza, who leads the Robotics and Perception Group at the University of Zurich and the NCCR Robotics Rescue Robotics Grand Challenge. “This is very difficult both for humans and for machines. Expert human pilots can reach this level after years of perseverance and training. But machines still struggle.”

The AI algorithm learns to fly in the real world from a simulated expert In a new study, Scaramuzza and his team have trained an autonomous quadrotor to fly through previously unseen environments such as forests, buildings, ruins and trains, keeping speeds of up to 40 km/h and without crashing into trees, walls or other obstacles. All this was achieved relying only on the quadrotor’s on-board cameras and computation.

The drone’s neural network learned to fly by watching a sort of “simulated expert” – an algorithm that flew a computer-generated drone through a simulated environment full of complex obstacles. At all times, the algorithm had complete information on the state of the quadrotor and readings from its sensors, and could rely on enough time and computational power to always find the best trajectory.

Such a “simulated expert” could not be used outside of simulation, but its data were used to teach the neural network how to predict the best trajectory based only on the data from the sensors. This is a considerable advantage over existing systems, which first use sensor data to create a map of the environment and then plan trajectories within the map – two steps that require time and make it impossible to fly at high-speeds.

No exact replica of the real world needed After being trained in simulation, the system was tested in the real world, where it was able to fly in a variety of environments without collisions at speeds of up to 40 km/h. “While humans require years to train, the AI, leveraging high-performance simulators, can reach comparable navigation abilities much faster, basically overnight,” says Antonio Loquercio, a PhD student and co-author of the paper. “Interestingly these simulators do not need to be an exact replica of the real world. If using the right approach, even simplistic simulators are sufficient,” adds Elia Kaufmann, another PhD student and co-author.

The applications are not limited to quadrotors. The researchers explain that the same approach could be useful for improving the performance of autonomous cars, or could even open the door to a new way of training AI systems for operations in domains where collecting data is difficult or impossible, for example on other planets.

According to the researchers, the next steps will be to make the drone improve from experience, as well as to develop faster sensors that can provide more information about the environment in a smaller amount of time – thus allowing drones to fly safely even at speeds above 40 km/h.

Literature * Learning High-speed Flight in the Wild. Antonio Loquercio, Elia Kaufmann, René Ranftl, Matthias Müller, Vladlen Koltun and Davide Scaramuzza. Science Robotics, October 6, 2021. DOI: 10.1126/scirobotics.abg5810

An open-source version of the paper can be found here.

Media contacts Prof. Dr. Davide Scaramuzza – Robotics and Perception Group

Department of Informatics

University of Zurich

Phone +41 44 635 24 09

E-mail: sdavide@ifi.uzh.ch

Antonio Loquercio – Robotics and Perception Group

Department of Informatics

University of Zurich

Phone +41 44 635 43 73

E-mail: loquercio@ifi.uzh.ch

Elia Kaufmann – Robotics and Perception Group

Institut für Informatik

Universität Zürich

Tel. +41 44 635 43 73

E-Mail: ekaufmann@ifi.uzh.ch

Media Relations University of Zurich

Phone +41 44 634 44 67

E-mail: mediarelations@kommunikation.uzh.ch