Robots expand their umwelt

Empowering robots with human-like perception

Excerpt: "A new Duke-developed AI system fuses vision, vibrations, touch and its own body states to help robots understand and move through difficult in-the-wild environments.

The wealth of information provided by our senses that allows our brain to navigate the world around us is remarkable. Touch, smell, hearing, and a strong sense of balance are crucial to making it through what to us seem like easy environments such as a relaxing hike on a weekend morning.

An innate understanding of the canopy overhead helps us figure out where the path leads. The sharp snap of branches or the soft cushion of moss informs us about the stability of our footing. The thunder of a tree falling or branches dancing in strong winds lets us know of potential dangers nearby.

Robots, in contrast, have long relied solely on visual information such as cameras or lidar to move through the world.

Outside of Hollywood, multisensory navigation has long remained challenging for machines. The forest, with its beautiful chaos of dense undergrowth, fallen logs and ever-changing terrain, is a maze of uncertainty for traditional robots."