A team of researchers from the University of Bonn, Germany, have won five million US dollars at the ANA Avatar XPRIZE competition in Long Beach, USA. The final, hosted by All Nippon Airways (ANA), included 17 teams from ten countries. These teams were selected from 99 initial teams in a multi-stage process. The ANA Avatar XPRIZE competition is the most highly endowed competition in robotics to date.
The German team started their journey to victory when they scored the most points during the semi-finals last September. The aim of the competition was to develop robotic systems allowing people to virtually put themselves in a different place.
Their robot works via an operator station and an avatar robot connected via the internet. Sensors in the avatar capture the environment, and the results are displayed in the operator station allowing the user to have the feeling of being in a remote location. Then, the user’s movements are captured and transferred to the avatar robot. This allows the user to move around, use objects and communicate through speech, facial expressions, and gestures.
To add an extra challenge, during the competition, the avatars were controlled by members of the jury, who only had a few minutes to familiarise themselves with the system. “Therefore, intuitive usability was an important objective in system design,” said Prof. Dr. Sven Behnke of Team NimbRo, head of the Autonomous Intelligent Systems group at the University of Bonn.
During the finals, there was one qualification step, and then two competition runs. During these sessions, the jury had to solve ten tasks, which were part of a mission set on a distant planet. Tasks included, for example, communicating with a human, operating a switch, using a power drill, and identifying a stone by touch. In addition to these tasks, the judges also assess the overall quality of the experience.
In the two competition runs, not only NimbRo obtained full points, but it was almost twice as fast as the second team from France. For their efforts, the NimbRo team received the grand prize of five million US dollars. “My fantastic team developed a powerful avatar system for the complex tasks of the competition and successfully solved all problems that arose,” said Prof. Behnke.
The NimbRo avatar was developed over three years at the Autonomous Intelligent Systems group of the Institute of Computer Science VI – Intelligent Systems and Robotics at the University of Bonn, led by Prof. Dr. Sven Behnke. The system has a human-like upper body with two arms and five fingers in each hand. Located on the head, the avatar has a wide-angle camera, a microphone, and a display for the user’s face.
The remote scene is transmitted to the user through 3D glasses, which can also detect the user’s movements and transmit them back to the avatar. This allows the user to move around freely and see what’s in front of the avatar from different angles, which contributes to the feeling of immersion.
The user also feels when the avatar touches something via force-movement sensors in the wrist. This allows the robotic arms to sense the user’s hand movements, and hand exoskeletons mediate finger movements. Finger grip is felt by motor currents and transmitted to the user’s fingers, including fingertips.
The avatar can move in all directions and turn on the spot, with direction and speed controlled by the user’s foot. Glasses on the avatar are equipped with cameras to capture eye movement and facial expressions of the user for a live facial animation displayed on the avatar’s head.
This type of robot has endless applications, from helping people to manage their lives, to telemedicine and access to hazardous environments. “Avatar systems have great potential to overcome the spatial separation between people and could be as widespread in a few years as video conferencing systems are today,” concluded Prof. Behnke.
Videos of the final competition runs by Team NimbRo from the University of Bonn:
https://youtu.be/pxblVcN606E
https://youtu.be/8AwgGSpcAe8
Information about the avatar system: https://www.ais.uni-bonn.de/nimbro/AVATAR
Information about the competition: https://www.xprize.org/prizes/avatar