AI-Enabled Autonomous Service Robots

Localization of human speakers is highly important for natural human-robot interaction. However, in real-life environments such as public places, localization is a great challenge due to strong reverberation, multiple speakers, and noise. We are developing speaker localization algorithm that is robust to reverberation, for spherical arrays applied to arrays around a robot head, with a real-time version for various real-life acoustic conditions.

In noisy environments which include reverberation, speech recognition rate tends to significantly decrease. Microphone arrays with multiple channel signal processing offer a successful approach for speech enhancement and can improve speech enhancement under noise and reverberation.

Producing a high quality, globally consistent map in real-time with the help of one or more robots, by using semantic labeling of the map geometries, so the robot can plan its course in dynamic and complex environments. By adding layers to the map, robots can contribute and share their local information collaboratively, share their local view and update the maps, resulting in lower energy consumption and serving more customers in less time.

In public places, hand gestures are complementary mean to voice commands, when calling robots for assistance. Voice commands might constitute a limitation, especially when the robot doesn't "speak" the language of the random user, and therefore for environments such as airports, independent gestures interaction is useful for different tasks according to location demographic.

Exploring the effect of different emotions on the way people interact with a robot. The environment is used as a mediator, and changing the environmental conditions is expected to yield different emotions among participants. These emotions are detected by an analysis of the tone of voice, accompanied with physiological measurements and self-reported measures.

Real-time interface to the video system used by the robot, based on cumulative video and speech samples that simulate real interactions scenarios, in terms of conversation culture, language, subjects, length and real-time dialogue.

A public space is often supported by a variety of on-screen content, which people interact with. Developing a robotic system which responds to video content with robotic reactions, both verbal and non-verbal is a new media experience, which falls in-between content and the consumer and achieves a much higher level of engagement. This new way of experiencing media could significantly shift content creation and media delivery with the robots’ behavior implemented as a separate media channel accompanying the video, but experienced in the real-world.

Non-verbal signal detection of “time to interrupt”​. Verbal dialog is regulated using non-verbal signals, including pauses, head movements, eye gaze, and more. This Machine Learning project, models dialogs based on non-verbal signals, so a robot that can intelligently produce the right non-verbal signals to help make interruptions more fluent and acceptable.

Pause-and-Restart. A common failure of a robot’s interaction is completing or starting an action or sentence when no longer necessary, or not knowing how to re-start an interaction. We're building a model for socially natural pauses and continuation.