MINDMAPS: Semantic Topometric Maps as Models of the Environment for Intelligent Robots

(Sep’24–Aug’27)
National Project

Intelligent mobile robots, much like humans, require a detailed and comprehensive map of their environment to make grounded decisions, reason effectively, and interact naturally with people. These models, denominated maps in robotics, are an essential component of the robot’s cognitive abilities. This project aims to investigate how advanced deep learning techniques applied to images, such as object recognition, scene classification, and natural language processing (NLP), can be used to create such maps. Our research will focus on several issues. First, by combining data from depth sensors and cameras, we will tackle how to transition from image object recognition to situating semantic objects in 3D space while maintaining spatial and temporal coherence across different locations, times, and sensors. This is challenging since we need to cope with the uncertainty in the robot pose and the sensor measurements, occlusions, partial views, illumination changes, dynamic objects, etc. Once we have semantic entities in space, we want to organize them into topological structures based on graphs of objects and places to overcome the drawbacks of a monolithic world representation (with a single reference frame) when the environment grows. These drawbacks include spatial inconsistencies, precision drift, path/task planning inefficiency, or the high computational complexity of data analysis. We refer to such world models as Semantic Topometric maps or STM-maps. Furthermore, we will develop a software architecture distributed between the robot, the edge and the cloud. This architecture will help in the efficient creation, maintenance, and exploitation of STM-maps, which could be fed into state-of-the-art world representation frameworks like Unity 3D for real-time visualization and simulation. This is a bold initiative that could make a first step towards robotic digital-twin models.

REFERENCE: PID2023-148191NB-I00
FUNDED BY: Ministerio de Ciencia e Innovación
PERIOD: Sep 2024 – Aug 2027
PRINCIPAL RESEARCHERS: JAVIER GONZÁLEZ JIMÉNEZ and JOSÉ RAÚL RUIZ SARMIENTO
INSTITUTION: University of Málaga