Dr. José Raúl Ruiz Sarmiento
Associate Professor
Address: Dpto. Ingenieria de Sistemas y Automatica, E.T.S.I. Informatica – Telecomunicacion, Universidad de Malaga, Campus Universitario de Teatinos, 29071 Malaga, Spain
Phone: +34 952 13 28 94
E-mail: jotaraul [at] uma.es
Research interests: Computer vision, Object/place recognition, Semantic Maps, Probabilistic Graphical Models, Data science, Deep learning
 |  |  |  |
Contents
Short bio
I was born in Málaga, Spain, in 1984. I received the B.Sc. in Computer Science Engineering from the University of Málaga in July 2009, and one year later the M.Sc. in Mechatronics. Before that, in September 2008 I joined the MAPIR group. Since then, I was involved in different projects like the TCS national project or the European one ExCITE, until in January 2012 I obtained an FPU grant from the Spanish government — a great oportunity to take advantage of 4 years as PhD. student.
My research was in the field of cognitive robotics, specifically in perception. I attended conferences, published in journals, performed a research stay in Osnabrück (Germany) under the supervision of Prof. Joachim Hertberg, and attended a winter school in Orebrö (Sweeden), among other cool things. In November 2016 I completed my PhD in Computer Science at the University of Málaga.
In 2018, I got a competitive post-doc contract from the same university, and in 2019, I did a stay in the Active Vision Laborator of the University of Oxford under the supervision of Prof. Victor Adrian Prisacariu. In 2020, I started my path as an Assistant Professor and became an Associate Professor in 2025. Nowadays, I continue researching in cognitive robotics, participating in national and international projects, and collaborating with companies. Let’s imagine a better future and make it real! 😊
My brief CV can be found [here].
Academic background
- PhD in Computer Science Engineering with honors. University of Málaga, November 2016.
- M.Sc in Mechatronics. University of Málaga, July 2010.
- B.Sc in Computer Science Engineering. University of Málaga, July 2009.
Awards
- OMRON award to the best work in the Computer Vision Thematic Group in the Jornadas de Automática 2024 conference for Large models for semantic mapping in mobile robotics.
- ROBOTNIK award to the best PhD thesis in Robotics, presented in the Jornadas de Automática 2017, for the thesis entitled Probabilistic Techniques in Semantic Mapping for Mobile Robots.
- Finalist of the IBIMA awards to the best 2017 paper for the work Robot@Home, a Robotic Dataset for Semantic Mapping of Home Environments.
- PAAMS’11 award for scientific excellence for the work Improving Human Face Detection through ToF Cameras for Ambient Intelligence Applications
- INFAIMON 2011 award to the best work on Applications of Computer Vision to Automation for Cámaras basadas en tiempo de vuelo. Uso en la mejora de métodos de detección de caras.
Scientific production (publications)
- Journal papers: 24
- Conference papers: 39
- Book chapters: 7
- Technical reports: 1
Mentoring
- Supervised PhD Students:
- Jesús Moncada Ramírez (ongoing, co-supervised by Javier Gonzalez Jimenez)
- Gregorio Ambrosio Cestero (ongoing, co-supervised by Javier Gonzalez Jimenez)
- Andrés Galeote Luque (2024, co-supervised by Javier Gonzalez Jimenez)
- Supervised M.Sc. Students: 3
- Supervised B.Sc. Students: 22
Participation in Projects
- MINDMAPS: Semantic Topometric Maps as Models of the Environment for Intelligent Robots (PID2023-148191NB-I00), Spanish Ministry of Science and Innovation, Sept’2024-Aug’2027 (Principal Investigator) Ongoing
- VOXELAND: Towards an effective voxelized representation of the workspace of mobile robots (JA.B1-09). University of Málaga. Jan’2024-Dec’2027 (Principal Investigator) Ongoing
- AI system for the automatic management of green areas (AEI-010500-2023-207). Spanish Ministry of Industry, Comerce and Turism, Innovative Business Clusters -2023 MRR, Spanish call, June’2023-Apr’2024 (Research member)
- 5GVec (OTRI-8.06/5.56.6558), TUPL SPAIN S.L. Apr’2023-Dec’2023.
- ARPEGGIO: Advanced Robotic Perception for Mapping and Localization (PID2020-117057GB-I00), Spanish Ministry of Science and Innovation, Sept’2021-Feb’2025 (Research member)
- HOUNDBOT: Creación de mapas de gas y localización de fuente con un robot móvil (P20_01302), Junta de Andalucía, Oct’21-Sept’23 (Research member)
- WISER: Building and exploiting semantic maps by mobile robots (DPI2017-84827-R). MINECO, Spanish call. Jan’18-Dec’20. (Research member)
- 6im: Laboratorio Exaescalable para Inteligencia Artificial y Modelado Numérico (Project ID: EQC2018-004571-P). MINECO, Spanish call. Nov’2018-Oct’20· (Research member)
- MoveCare: Multiple-actOrs Virtual Empathic CARgiver for the Elder (Project ID: 732158). Call H2020-ICT-2016-2017. Jan’2017-Dec’19· (Contract staff)
- SiMoDiM: Monitorización y Diagnóstico para la transformación digital del mantenimiento predictivo y correctivo de la maquinaria industrial del proceso de laminación en caliente (OTRI-8.06/5.56.4826). Acerinox Europa S.A.U. Mar’17-Mar’18. (Research member)
- IRO: Mejora de la capacidad sensorial y autónoma de robots mediante olfato (2012-TEP-530). Junta de Andalucia, Spanish call. Jan’14-Jan’18. (Research member)
- PROMOVE: Advances in mobile robotics for promoting independent life of elders (DPI2014-55826-R). MINECO, Spanish call. Jan’15-Dec’17. (Contract staff)
- Taroth: Nuevos desarrollos hacia un robot en casa (DPI2011-25483). CICYT, Spanish call. Jan’12-Dec’14. (Research member)
- ExCite: Enabling SoCial Interaction Through Embodiment (PS09/02678). AAL (Ambient Assited Living). Dec’09-Dec’12. (Contract staff)
- TCS: Tunnel Continuous Setout (OTRI-8.06/5.56.3153). SACYR S.A.U. Nov’08-Apr’10. (Contract staff)
Complete list of publications
Journals (26)
2026
M. Decena-Giménez, P. Ojeda, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Open-Vocabulary Instance Segmentation for Scene Understanding in Mobile Robots
MDPI Robotics, vol. 15, iss. 9, 2026. Q2 · T2
@article{decena2026,
title = {Open-Vocabulary Instance Segmentation for Scene Understanding in Mobile Robots},
author = {Decena-Giménez, Macorís and Ojeda, Pepe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://www.mdpi.com/2218-6581/15/9/175},
doi = {https://doi.org/10.3390/robotics15090175},
year = {2026},
date = {2026-01-01},
urldate = {2026-01-01},
journal = {MDPI Robotics},
volume = {15},
issue = {9},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J.L. Matez-Bandera, P. Ojeda, J. Monroy, J. Gonzalez-Jimenez, J.R. Ruiz-Sarmiento
Robotics and Autonomous Systems, vol. 205, pp. 105668, 2026. Q1 · T1
@article{matezbandera2026,
title = {Voxeland: Probabilistic Instance-Aware Semantic Mapping with Evidence-based Uncertainty Quantification},
author = {Matez-Bandera, J. L and Ojeda, Pepe and Monroy, Javier and Gonzalez-Jimenez, Javier and Ruiz-Sarmiento, Jose-Raul},
url = {https://arxiv.org/abs/2411.08727},
doi = {10.1016/j.robot.2026.105668},
year = {2026},
date = {2026-01-01},
urldate = {2026-01-01},
journal = {Robotics and Autonomous Systems},
volume = {205},
pages = {105668},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2025
J. Moncada-Ramirez, J.L. Matez-Bandera, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Agentic Workflows for Improving Large Language Model Reasoning in Robotic Object-Centered Planning
Robotics, vol. 14, no. 3, 2025, ISSN: 2218-6581. Q2 · T2
@article{moncada2025robotics,
title = {Agentic Workflows for Improving Large Language Model Reasoning in Robotic Object-Centered Planning},
author = {Moncada-Ramirez, Jesús and Matez-Bandera, J. L and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://www.mdpi.com/2218-6581/14/3/24},
doi = {10.3390/robotics14030024},
issn = {2218-6581},
year = {2025},
date = {2025-01-01},
journal = {Robotics},
volume = {14},
number = {3},
abstract = {Large Language Models (LLMs) provide cognitive capabilities that enable robots to interpret and reason about their workspace, especially when paired with semantically rich representations like semantic maps. However, these models are prone to generating inaccurate or invented responses, known as hallucinations, that can produce an erratic robotic operation. This can be addressed by employing agentic workflows, structured processes that guide and refine the model’s output to improve response quality. This work formally defines and qualitatively analyzes the impact of three agentic workflows (LLM Ensemble, Self-Reflection, and Multi-Agent Reflection) on enhancing the reasoning capabilities of an LLM guiding a robotic system to perform object-centered planning. In this context, the LLM is provided with a pre-built semantic map of the environment and a query, to which it must respond by determining the most relevant objects for the query. This response can be used in a multitude of downstream tasks. Extensive experiments were carried out employing state-of-the-art LLMs and semantic maps generated from the widely-used datasets ScanNet and SceneNN. The results show that agentic workflows significantly enhance object retrieval performance, especially in scenarios requiring complex reasoning, with improvements averaging up to 10% over the baseline.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Large Language Models (LLMs) provide cognitive capabilities that enable robots to interpret and reason about their workspace, especially when paired with semantically rich representations like semantic maps. However, these models are prone to generating inaccurate or invented responses, known as hallucinations, that can produce an erratic robotic operation. This can be addressed by employing agentic workflows, structured processes that guide and refine the model’s output to improve response quality. This work formally defines and qualitatively analyzes the impact of three agentic workflows (LLM Ensemble, Self-Reflection, and Multi-Agent Reflection) on enhancing the reasoning capabilities of an LLM guiding a robotic system to perform object-centered planning. In this context, the LLM is provided with a pre-built semantic map of the environment and a query, to which it must respond by determining the most relevant objects for the query. This response can be used in a multitude of downstream tasks. Extensive experiments were carried out employing state-of-the-art LLMs and semantic maps generated from the widely-used datasets ScanNet and SceneNN. The results show that agentic workflows significantly enhance object retrieval performance, especially in scenarios requiring complex reasoning, with improvements averaging up to 10% over the baseline.
2024
A. Jaenal, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Engineering Applications of Artificial Intelligence, vol. 127, pp. 107365, 2024, ISSN: 0952-1976. Q1 · T1
@article{alberto2024eaai,
title = {MachNet, a general Deep Learning architecture for Predictive Maintenance within the industry 4.0 paradigm},
author = {Jaenal, Alberto and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://www.sciencedirect.com/science/article/pii/S095219762301549X},
doi = {10.1016/j.engappai.2023.107365},
issn = {0952-1976},
year = {2024},
date = {2024-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {127},
pages = {107365},
abstract = {In the Industry 4.0 era, a myriad of sensors of diverse nature (temperature, pressure, etc.) is spreading throughout the entire value chain of industries, being potentially exploitable for multiple purposes, such as Predictive Maintenance (PdM): the just-in-time maintenance of industrial assets, which results in reduced operating costs, increased operator safety, etc. Nowadays, industrial processes require to be highly configurable, in order to proactively adapt their operation to diverse factors such as user needs, product updates or supply chain uncertainties. This limits current Industry 4.0-PdM solutions, typically consisting of ad-hoc developments intended for specific scenarios, i.e. they are designed to operate under certain conditions (configurations, employed sensors, etc.), being unable to manage changes in their setup. This paper presents a general Deep Learning (DL) architecture, MachNet, which deals with such heterogeneity and is able to address PdM problems of a diverse nature. The modularity of the proposed architecture enables it to deal with an arbitrary number of sensors of different types, also allowing the integration of prior information (age of assets, material type, etc.), which clearly affects performance and is often neglected. In practice, our architecture effortlessly adapts to the assets’ specifications and to different PdM problems. That is, MachNet becomes an architectural template that can be instantiated for a given scenario. We tested our proposal in two different PdM-related problems: Health State (HS) and Remaining-useful-Life (RuL) estimation, achieving in both cases comparable or superior performance to other state-of-the-art approaches, with the additional advantage of the generality that MachNet offers.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In the Industry 4.0 era, a myriad of sensors of diverse nature (temperature, pressure, etc.) is spreading throughout the entire value chain of industries, being potentially exploitable for multiple purposes, such as Predictive Maintenance (PdM): the just-in-time maintenance of industrial assets, which results in reduced operating costs, increased operator safety, etc. Nowadays, industrial processes require to be highly configurable, in order to proactively adapt their operation to diverse factors such as user needs, product updates or supply chain uncertainties. This limits current Industry 4.0-PdM solutions, typically consisting of ad-hoc developments intended for specific scenarios, i.e. they are designed to operate under certain conditions (configurations, employed sensors, etc.), being unable to manage changes in their setup. This paper presents a general Deep Learning (DL) architecture, MachNet, which deals with such heterogeneity and is able to address PdM problems of a diverse nature. The modularity of the proposed architecture enables it to deal with an arbitrary number of sensors of different types, also allowing the integration of prior information (age of assets, material type, etc.), which clearly affects performance and is often neglected. In practice, our architecture effortlessly adapts to the assets’ specifications and to different PdM problems. That is, MachNet becomes an architectural template that can be instantiated for a given scenario. We tested our proposal in two different PdM-related problems: Health State (HS) and Remaining-useful-Life (RuL) estimation, achieving in both cases comparable or superior performance to other state-of-the-art approaches, with the additional advantage of the generality that MachNet offers.
2023
A. Galeote-Luque, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
GND-LO: Ground Decoupled 3D Lidar Odometry Based on Planar Patches
IEEE Robotics and Automation Letters, vol. 8, no. 11, pp. 6923 - 6930, 2023. Q2 · T1
@article{galeoteluque2023,
title = {GND-LO: Ground Decoupled 3D Lidar Odometry Based on Planar Patches},
author = {Galeote-Luque, Andres and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://ieeexplore.ieee.org/abstract/document/10243099},
doi = {10.1109/LRA.2023.3313057},
year = {2023},
date = {2023-09-01},
journal = {IEEE Robotics and Automation Letters},
volume = {8},
number = {11},
pages = {6923 - 6930},
abstract = {We present a new fast Ground Decoupled 3D Lidar Odometry (GND-LO) method. The particularity of GND-LO is that it takes advantage of the distinct spatial layout found in urban settings to efficiently recover the lidar movement in a decoupled manner. For that, the input scans are reduced to a set of planar patches extracted from the flat surfaces of the scene, found aplenty in these scenarios. These patches can be labeled as either belonging to the ground or walls, decoupling the estimation into two steps. First, the ground planes from each scan, clustered from the ground patches, are registered. Then, the motion estimation is completed by minimizing the distance between the wall patches and their corresponding points from the other scan, whose pairing is iteratively updated. GND-LO has demonstrated to perform both precisely and efficiently beating state-of-the-art approaches. Concretely, experiments on the popular KITTI dataset show that our proposal outperforms its competitors by reducing the average drift by 19% in translation and 4% in rotation. This is achieved by running in real-time without needing GPU or optimized multi-threading, as it is commonplace in the literature.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a new fast Ground Decoupled 3D Lidar Odometry (GND-LO) method. The particularity of GND-LO is that it takes advantage of the distinct spatial layout found in urban settings to efficiently recover the lidar movement in a decoupled manner. For that, the input scans are reduced to a set of planar patches extracted from the flat surfaces of the scene, found aplenty in these scenarios. These patches can be labeled as either belonging to the ground or walls, decoupling the estimation into two steps. First, the ground planes from each scan, clustered from the ground patches, are registered. Then, the motion estimation is completed by minimizing the distance between the wall patches and their corresponding points from the other scan, whose pairing is iteratively updated. GND-LO has demonstrated to perform both precisely and efficiently beating state-of-the-art approaches. Concretely, experiments on the popular KITTI dataset show that our proposal outperforms its competitors by reducing the average drift by 19% in translation and 4% in rotation. This is achieved by running in real-time without needing GPU or optimized multi-threading, as it is commonplace in the literature.
G. Ambrosio, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
The Robot@Home2 dataset: A new release with improved usability tools
SoftwareX, vol. 23, pp. 101490, 2023, ISSN: 2352-7110. Q2 · T2
@article{goyo2023softwarex,
title = {The Robot@Home2 dataset: A new release with improved usability tools},
author = {Ambrosio, Gregorio and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.1016/j.softx.2023.101490},
issn = {2352-7110},
year = {2023},
date = {2023-07-01},
journal = {SoftwareX},
volume = {23},
pages = {101490},
abstract = {Released in 2017, Robot@Home is a dataset captured by a mobile robot during indoor navigation sessions in apartments. This paper presents Robot@Home2, an enhanced version of the Robot@Home dataset, aimed at improving usability and functionality for developing and testing mobile robotics and computer vision algorithms. Robot@Home2 consists of three main components. Firstly, a relational database that states the contextual information and data links, compatible with Standard Query Language. Secondly,a Python package for managing the database, including downloading, querying, and interfacing functions. Finally, learning resources in the form of Jupyter notebooks, runnable locally or on the Google Colab platform, enabling users to explore the dataset without local installations. These freely available tools are expected to enhance the ease of exploiting the Robot@Home dataset and accelerate research in computer vision and robotics.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Released in 2017, Robot@Home is a dataset captured by a mobile robot during indoor navigation sessions in apartments. This paper presents Robot@Home2, an enhanced version of the Robot@Home dataset, aimed at improving usability and functionality for developing and testing mobile robotics and computer vision algorithms. Robot@Home2 consists of three main components. Firstly, a relational database that states the contextual information and data links, compatible with Standard Query Language. Secondly,a Python package for managing the database, including downloading, querying, and interfacing functions. Finally, learning resources in the form of Jupyter notebooks, runnable locally or on the Google Colab platform, enabling users to explore the dataset without local installations. These freely available tools are expected to enhance the ease of exploiting the Robot@Home dataset and accelerate research in computer vision and robotics.
2022
A. Galeote-Luque, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Efficient 3D Lidar Odometry Based on Planar Patches
Sensors, vol. 22, no. 18, 2022, ISSN: 1424-8220. Q2 · T1
@article{andres2022efficient,
title = {Efficient 3D Lidar Odometry Based on Planar Patches},
author = {Galeote-Luque, Andres and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://www.mdpi.com/1424-8220/22/18/6976},
doi = {10.3390/s22186976},
issn = {1424-8220},
year = {2022},
date = {2022-01-01},
journal = {Sensors},
volume = {22},
number = {18},
abstract = {In this paper we present a new way to compute the odometry of a 3D lidar in real-time. Due to the significant relation between these sensors and the rapidly increasing sector of autonomous vehicles, 3D lidars have improved in recent years, with modern models producing data in the form of range images. We take advantage of this ordered format to efficiently estimate the trajectory of the sensor as it moves in 3D space. The proposed method creates and leverages a flatness image in order to exploit the information found in flat surfaces of the scene. This allows for an efficient selection of planar patches from a first range image. Then, from a second image, keypoints related to said patches are extracted. This way, our proposal computes the ego-motion by imposing a coplanarity constraint between pairs <point, plane> whose correspondences are iteratively updated. The proposed algorithm is tested and compared with state-of-the-art ICP algorithms. Experiments show that our proposal, running on a single thread, can run 5× faster than a multi-threaded implementation of GICP, while providing a more accurate localization. A second version of the algorithm is also presented, which reduces the drift even further while needing less than half of the computation time of GICP. Both configurations of the algorithm run at frame rates common for most 3D lidars, 10 and 20 Hz on a standard CPU.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this paper we present a new way to compute the odometry of a 3D lidar in real-time. Due to the significant relation between these sensors and the rapidly increasing sector of autonomous vehicles, 3D lidars have improved in recent years, with modern models producing data in the form of range images. We take advantage of this ordered format to efficiently estimate the trajectory of the sensor as it moves in 3D space. The proposed method creates and leverages a flatness image in order to exploit the information found in flat surfaces of the scene. This allows for an efficient selection of planar patches from a first range image. Then, from a second image, keypoints related to said patches are extracted. This way, our proposal computes the ego-motion by imposing a coplanarity constraint between pairs <point, plane> whose correspondences are iteratively updated. The proposed algorithm is tested and compared with state-of-the-art ICP algorithms. Experiments show that our proposal, running on a single thread, can run 5× faster than a multi-threaded implementation of GICP, while providing a more accurate localization. A second version of the algorithm is also presented, which reduces the drift even further while needing less than half of the computation time of GICP. Both configurations of the algorithm run at frame rates common for most 3D lidars, 10 and 20 Hz on a standard CPU.
J.L. Matez-Bandera, D. Fernandez-Chaves, J.R. Ruiz-Sarmiento, J. Monroy, N. Petkov, J. Gonzalez-Jimenez
LTC-Mapping, Enhancing Long-term Consistency of Object-oriented Semantic Maps in Robotics
MDPI Sensors, vol. 22, no. 14, 2022, ISSN: 1424-8220. Q2 · T2
@article{matez_mdpi_2022,
title = {LTC-Mapping, Enhancing Long-term Consistency of Object-oriented Semantic Maps in Robotics},
author = {Matez-Bandera, J. L and Fernandez-Chaves, David and Ruiz-Sarmiento, J. R. and Monroy, Javier and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2022/2022_matez_LTCmapping.pdf},
doi = {10.3390/s22145308},
issn = {1424-8220},
year = {2022},
date = {2022-01-01},
journal = {MDPI Sensors},
volume = {22},
number = {14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
D. Fernandez-Chaves, J.R. Ruiz-Sarmiento, A. Jaenal, N. Petkov, J. Gonzalez-Jimenez
Expert Systems with Applications, vol. 208, pp. 117970, 2022, ISSN: 0957-4174. Q1 · T1
@article{fernandez2022eswa,
title = {Robot@VirtualHome, an ecosystem of virtual environments and tools for realistic indoor robotic simulation},
author = {Fernandez-Chaves, David and Ruiz-Sarmiento, J. R. and Jaenal, Alberto and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://www.sciencedirect.com/science/article/pii/S0957417422012015},
doi = {10.1016/j.eswa.2022.117970},
issn = {0957-4174},
year = {2022},
date = {2022-01-01},
journal = {Expert Systems with Applications},
volume = {208},
pages = {117970},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2021
S.F. Baltanas-Molero, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Improving the Head Pose Variation Problem in Face Recognition for Mobile Robots
Sensors, vol. 21, no. 2, pp. 659, 2021, ISSN: 1424-8220. Q2 · T1
@article{baltanas2021face,
title = {Improving the Head Pose Variation Problem in Face Recognition for Mobile Robots},
author = {Baltanas-Molero, Samuel-Felipe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://www.mdpi.com/1424-8220/21/2/659},
doi = {10.3390/s21020659},
issn = {1424-8220},
year = {2021},
date = {2021-01-01},
journal = {Sensors},
volume = {21},
number = {2},
pages = {659},
abstract = {Face recognition is a technology with great potential in the field of robotics, due to its prominent role in human-robot interaction (HRI). This interaction is a keystone for the successful deployment of robots in areas requiring a customized assistance like education and healthcare, or assisting humans in everyday tasks. These unconstrained environments present additional difficulties for face recognition, extreme head pose variability being one of the most challenging. In this paper, we address this issue and make a fourfold contribution. First, it has been designed a tool for gathering an uniform distribution of head pose images from a person, which has been used to collect a new dataset of faces, both presented in this work. Then, the dataset has served as a testbed for analyzing the detrimental effects this problem has on a number of state-of-the-art methods, showing their decreased effectiveness outside a limited range of poses. Finally, we propose an optimization method to mitigate said negative effects by considering key pose samples in the recognition system’s set of known faces. The conducted experiments demonstrate that this optimized set of poses significantly improves the performance of a state-of-the-art, cutting-edge system based on Multitask Cascaded Convolutional Neural Networks (MTCNNs) and ArcFace.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Face recognition is a technology with great potential in the field of robotics, due to its prominent role in human-robot interaction (HRI). This interaction is a keystone for the successful deployment of robots in areas requiring a customized assistance like education and healthcare, or assisting humans in everyday tasks. These unconstrained environments present additional difficulties for face recognition, extreme head pose variability being one of the most challenging. In this paper, we address this issue and make a fourfold contribution. First, it has been designed a tool for gathering an uniform distribution of head pose images from a person, which has been used to collect a new dataset of faces, both presented in this work. Then, the dataset has served as a testbed for analyzing the detrimental effects this problem has on a number of state-of-the-art methods, showing their decreased effectiveness outside a limited range of poses. Finally, we propose an optimization method to mitigate said negative effects by considering key pose samples in the recognition system’s set of known faces. The conducted experiments demonstrate that this optimized set of poses significantly improves the performance of a state-of-the-art, cutting-edge system based on Multitask Cascaded Convolutional Neural Networks (MTCNNs) and ArcFace.
J.R. Ruiz-Sarmiento, S.F. Baltanas-Molero, J. Gonzalez-Jimenez
Jupyter Notebooks in Undergraduate Mobile Robotics Courses: Educational Tool and Case Study
Applied Sciences, vol. 11, no. 3, pp. 917, 2021, ISSN: 2076-3417. Q2 · T2
@article{raul2021jupyter,
title = {Jupyter Notebooks in Undergraduate Mobile Robotics Courses: Educational Tool and Case Study},
author = {Ruiz-Sarmiento, J. R. and Baltanas-Molero, Samuel-Felipe and Gonzalez-Jimenez, Javier},
url = {https://www.mdpi.com/2076-3417/11/3/917},
doi = {10.3390/app11030917},
issn = {2076-3417},
year = {2021},
date = {2021-01-01},
journal = {Applied Sciences},
volume = {11},
number = {3},
pages = {917},
abstract = {Jupyter notebooks are recently emerging as a valuable pedagogical resource in academy, being adopted in educational institutions worldwide. This is mainly due to their ability to combine the expressiveness of traditional explanations from textbooks, with the interaction capabilities of software applications, which provides numerous benefits for both students and lecturers of different fields. One of the areas that could benefit from their adoption is such of mobile robotics, whose recent popularity has resulted in an increasing demand of trained practitioners with a solid theoretical and practical background. Therefore, there is a need of high quality learning materials adapted to modern tools and methodologies. With that in mind, this work explores how the introduction of Jupyter notebooks in undergraduate mobile robotic courses contributes to improve both teaching and learning experiences. For that, we first present a series of (publicly available) educational notebooks encompassing a variety of topics relevant for robotics, with a particular emphasis in the study of mobile robots and commonly used sensors. Those documents have been built from the ground up to take advantage of the Jupyter Notebook framework, bridging the typical gap between theoretical frame and interactive code. We also present a case study describing the notebooks usage in undergraduate courses at University of Málaga, including a discussion on the promising results and findings obtained.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Jupyter notebooks are recently emerging as a valuable pedagogical resource in academy, being adopted in educational institutions worldwide. This is mainly due to their ability to combine the expressiveness of traditional explanations from textbooks, with the interaction capabilities of software applications, which provides numerous benefits for both students and lecturers of different fields. One of the areas that could benefit from their adoption is such of mobile robotics, whose recent popularity has resulted in an increasing demand of trained practitioners with a solid theoretical and practical background. Therefore, there is a need of high quality learning materials adapted to modern tools and methodologies. With that in mind, this work explores how the introduction of Jupyter notebooks in undergraduate mobile robotic courses contributes to improve both teaching and learning experiences. For that, we first present a series of (publicly available) educational notebooks encompassing a variety of topics relevant for robotics, with a particular emphasis in the study of mobile robots and commonly used sensors. Those documents have been built from the ground up to take advantage of the Jupyter Notebook framework, bridging the typical gap between theoretical frame and interactive code. We also present a case study describing the notebooks usage in undergraduate courses at University of Málaga, including a discussion on the promising results and findings obtained.
D. Fernandez-Chaves, J.R. Ruiz-Sarmiento, N. Petkov, J. Gonzalez-Jimenez
ViMantic, a distributed robotic architecture for semantic mapping in indoor environments
International Journal of Knowledge-Based Systems, pp. 107440, 2021, ISSN: 0950-7051. Q1 · T1
@article{david2021kbs,
title = {ViMantic, a distributed robotic architecture for semantic mapping in indoor environments},
author = {Fernandez-Chaves, David and Ruiz-Sarmiento, J. R. and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2021/2021_fernandezchaves_KBS_vimantic.pdf
https://www.sciencedirect.com/science/article/pii/S0950705121007024},
doi = {10.1016/j.knosys.2021.107440},
issn = {0950-7051},
year = {2021},
date = {2021-01-01},
journal = {International Journal of Knowledge-Based Systems},
pages = {107440},
abstract = {Semantic maps augment traditional representations of robot workspaces, typically based on their geometry and/or topology, with meta-information about the properties, relations and functionalities of their composing elements. A piece of such information could be: fridges are appliances typically found in kitchens and employed to keep food in good condition. Thereby, semantic maps allow for the execution of high-level robotic tasks in an efficient way, e.g. “Hey robot, Store the leftover salad”. This paper presents ViMantic, a novel semantic mapping architecture for the building and maintenance of such maps, which brings together a number of features as demanded by modern mobile robotic systems, including: (i) a formal model, based on ontologies, which defines the semantics of the problem at hand and establishes mechanisms for its manipulation; (ii) techniques for processing sensory information and automatically populating maps with, for example, objects detected by cutting-edge CNNs; (iii) distributed execution capabilities through a client–server design, making the knowledge in the maps accessible and extendable to other robots/agents; (iv) a user interface that allows for the visualization and interaction with relevant parts of the maps through a virtual environment; (v) public availability, hence being ready to use in robotic platforms. The suitability of ViMantic has been assessed using Robot@Home, a vast repository of data collected by a robot in different houses. The experiments carried out consider different scenarios with one or multiple robots, from where we have extracted satisfactory results regarding automatic population, execution times, and required size in memory of the resultant semantic maps.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Semantic maps augment traditional representations of robot workspaces, typically based on their geometry and/or topology, with meta-information about the properties, relations and functionalities of their composing elements. A piece of such information could be: fridges are appliances typically found in kitchens and employed to keep food in good condition. Thereby, semantic maps allow for the execution of high-level robotic tasks in an efficient way, e.g. “Hey robot, Store the leftover salad”. This paper presents ViMantic, a novel semantic mapping architecture for the building and maintenance of such maps, which brings together a number of features as demanded by modern mobile robotic systems, including: (i) a formal model, based on ontologies, which defines the semantics of the problem at hand and establishes mechanisms for its manipulation; (ii) techniques for processing sensory information and automatically populating maps with, for example, objects detected by cutting-edge CNNs; (iii) distributed execution capabilities through a client–server design, making the knowledge in the maps accessible and extendable to other robots/agents; (iv) a user interface that allows for the visualization and interaction with relevant parts of the maps through a virtual environment; (v) public availability, hence being ready to use in robotic platforms. The suitability of ViMantic has been assessed using Robot@Home, a vast repository of data collected by a robot in different houses. The experiments carried out consider different scenarios with one or multiple robots, from where we have extracted satisfactory results regarding automatic population, execution times, and required size in memory of the resultant semantic maps.
2020
J.R. Ruiz-Sarmiento, J. Monroy, F.A. Moreno, C. Galindo, J.M. Bonelo, J. Gonzalez-Jimenez
A Predictive Model for the Maintenance of Industrial Machinery in the Context of Industry 4.0
Engineering Applications of Artificial Intelligence, vol. 87, 2020, ISSN: 0952-1976. Q1 · T1
@article{jotaraul_kbs_2020,
title = {A Predictive Model for the Maintenance of Industrial Machinery in the Context of Industry 4.0},
author = {Ruiz-Sarmiento, J. R. and Monroy, Javier and Moreno, Francisco-Angel and Galindo, Cipriano and Bonelo, J. M. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2020/2020-raul-A_Predictive_Model_for_the_Maintenance_of_Industrial_Machinery_in_the_Context_of_Industry_4.0.pdf},
doi = {10.1016/j.engappai.2019.103289},
issn = {0952-1976},
year = {2020},
date = {2020-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {87},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
F.A. Moreno, J. Monroy, J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Automatic Waypoint Generation to Improve Robot Navigation Through Narrow Spaces
MDPI Sensors, vol. 20, no. 1, 2020, ISSN: 1424-8220. Q1 · T1
@article{moreno_etal_2020_sensors,
title = {Automatic Waypoint Generation to Improve Robot Navigation Through Narrow Spaces},
author = {Moreno, Francisco-Angel and Monroy, Javier and Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2020/2020_moreno_waypoint_generation_sensors-20-00240.pdf},
doi = {10.3390/s20010240},
issn = {1424-8220},
year = {2020},
date = {2020-01-01},
journal = {MDPI Sensors},
volume = {20},
number = {1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2019
D. Zuñiga-Noël, J.R. Ruiz-Sarmiento, R. Gomez-Ojeda, J. Gonzalez-Jimenez
Automatic Multi-Sensor Extrinsic Calibration For Mobile Robots
IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2862-2869, 2019. Q1 · T1
@article{zuniga2019ral,
title = {Automatic Multi-Sensor Extrinsic Calibration For Mobile Robots},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
doi = {10.1109/LRA.2019.2922618},
year = {2019},
date = {2019-07-01},
journal = {IEEE Robotics and Automation Letters},
volume = {4},
number = {3},
pages = {2862-2869},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Monroy, F.A. Moreno, J. Gonzalez-Jimenez
Ontology-based conditional random fields for object recognition
International Journal of Knowledge-Based Systems, no. 168, pp. 100-108, 2019. Q1 · T1
@article{raul2018kbs,
title = {Ontology-based conditional random fields for object recognition},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Monroy, Javier and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019-raul-KBS-Ontology_based_Conditional_Random_Fields.pdf},
doi = {/10.1016/j.knosys.2019.01.005},
year = {2019},
date = {2019-03-01},
journal = {International Journal of Knowledge-Based Systems},
number = {168},
pages = {100-108},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J. Monroy, J.R. Ruiz-Sarmiento, F.A. Moreno, C. Galindo, J. Gonzalez-Jimenez
Olfaction, Vision, and Semantics for Mobile Robots. Results of the IRO Project
Sensors, vol. 19, no. 16: 3488, 2019. Q1 · T1
@article{2019MonroyIRO,
title = {Olfaction, Vision, and Semantics for Mobile Robots. Results of the IRO Project},
author = {Monroy, Javier and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019_monroy_SENSORS_IRO_project.pdf},
doi = {10.3390/s19163488},
year = {2019},
date = {2019-01-01},
journal = {Sensors},
volume = {19},
number = {16: 3488},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2018
M. Günther, J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez, J. Hertzberg
Context-aware 3D object anchoring for mobile robots
Robotics and Autonomous Systems, 2018. Q2 · T2
@article{GuentherRGGH2018ras,
title = {Context-aware 3D object anchoring for mobile robots},
author = {Günther, Martin and Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier and Hertzberg, Joachim},
doi = {10.1016/j.robot.2018.08.016},
year = {2018},
date = {2018-09-01},
journal = {Robotics and Autonomous Systems},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J. Monroy, J.R. Ruiz-Sarmiento, F.A. Moreno, F. Melendez-Fernandez, C. Galindo, J. Gonzalez-Jimenez
A Semantic-Based Gas Source Localization with a Mobile Robot Combining Vision and Chemical Sensing
Sensors, vol. 18, no. 12, 2018, ISSN: 1424-8220. Q1 · T1
@article{monroy2018sensors,
title = {A Semantic-Based Gas Source Localization with a Mobile Robot Combining Vision and Chemical Sensing},
author = {Monroy, Javier and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Melendez-Fernandez, Francisco and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_monroy_SENSORS_semantic_gas_source_localization.pdf},
doi = {10.3390/s18124174},
issn = {1424-8220},
year = {2018},
date = {2018-01-01},
journal = {Sensors},
volume = {18},
number = {12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2017
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
International Journal of Approximate Reasoning, vol. 83, pp. 434-451, 2017. Q2 · T2
@article{ruizsarmiento2017,
title = {A survey on learning approaches for Undirected Graphical Models. Application to scene object recognition},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-IJAR-A_Survey_on_Learning_Approaches_for_Undirected_Graphical_Models.pdf},
doi = {10.1016/j.ijar.2016.10.009},
year = {2017},
date = {2017-04-01},
journal = {International Journal of Approximate Reasoning},
volume = {83},
pages = {434-451},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Building Multiversal Semantic Maps for Mobile Robot Operation
Knowledege-Based Systems, vol. 119, pp. 257-272, 2017. Q1 · T1
@article{ruizsarmiento2017b,
title = {Building Multiversal Semantic Maps for Mobile Robot Operation},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-KBS-Building_Multiversal_Semantic_Maps.pdf},
doi = {10.1016/j.knosys.2016.12.016},
year = {2017},
date = {2017-03-01},
journal = {Knowledege-Based Systems},
volume = {119},
pages = {257-272},
abstract = {Semantic maps augment metric-topological maps with meta-information, i.e. l semantic knowledge aimed at the planning and execution of high-level robotic tasks. Semantic knowledge typically encodes human-like concepts, like types of objects and rooms, which are connected to sensory data when symbolic representations of percepts from the robot workspace are grounded to those concepts. Such a symbol grounding is usually carried out by algorithms that individually categorize each symbol and provide a crispy outcome – a symbol is either a member of a category or not. Such approach is valid for a variety of tasks, but it fails at: (i) dealing with the uncertainty inherent to the grounding process, and (ii) jointly exploiting the contextual relations among concepts (e.g. microwaves are usually in kitchens). This work provides a solution for probabilistic symbol grounding that overcomes these limitations. Concretely, we rely on Conditional Random Fields (CRFs) to model and exploit contextual relations, and to provide measurements about the uncertainty coming from the possible groundings in the form of beliefs (e.g. an object can be categorized (grounded) as a microwave or as a nightstand with beliefs 0.6 and 0.4, respectively). Our solution is integrated into a novel semantic map representation called Multiversal Semantic Map (MvSmap), which keeps the sets of different groundings, or universes, as instances of ontologies annotated with the obtained beliefs for their posterior exploitation. The suitability of our proposal has been proven with the Robot@Home dataset, a repository that contains challenging multi-modal sensory information gathered by a mobile robot in home environments.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Semantic maps augment metric-topological maps with meta-information, i.e. l semantic knowledge aimed at the planning and execution of high-level robotic tasks. Semantic knowledge typically encodes human-like concepts, like types of objects and rooms, which are connected to sensory data when symbolic representations of percepts from the robot workspace are grounded to those concepts. Such a symbol grounding is usually carried out by algorithms that individually categorize each symbol and provide a crispy outcome – a symbol is either a member of a category or not. Such approach is valid for a variety of tasks, but it fails at: (i) dealing with the uncertainty inherent to the grounding process, and (ii) jointly exploiting the contextual relations among concepts (e.g. microwaves are usually in kitchens). This work provides a solution for probabilistic symbol grounding that overcomes these limitations. Concretely, we rely on Conditional Random Fields (CRFs) to model and exploit contextual relations, and to provide measurements about the uncertainty coming from the possible groundings in the form of beliefs (e.g. an object can be categorized (grounded) as a microwave or as a nightstand with beliefs 0.6 and 0.4, respectively). Our solution is integrated into a novel semantic map representation called Multiversal Semantic Map (MvSmap), which keeps the sets of different groundings, or universes, as instances of ontologies annotated with the obtained beliefs for their posterior exploitation. The suitability of our proposal has been proven with the Robot@Home dataset, a repository that contains challenging multi-modal sensory information gathered by a mobile robot in home environments.
A. Orlandini, A. Kristoffersson, L. Almquist, P. Björkman, A. Cesta, G. Cortellessa, C. Galindo, J. Gonzalez-Jimenez, K. Gustafsson, A. Kiselev, A. Loufti, F. Melendez-Fernandez, M. Nilsson, L. Odens Hedman, E. Odontidou, J.R. Ruiz-Sarmiento, M. Scherlund, L. Tiberio, S.V. Rump, S. Coradeschi
ExCITE Project: A Review of Forty-two Months of Robotic Telepresence Technology Evolution
Presence: Teleoperators and Virtual Environments, 2017. Q4 · T3
@article{orlandini2017,
title = {ExCITE Project: A Review of Forty-two Months of Robotic Telepresence Technology Evolution},
author = {Orlandini, Andrea and Kristoffersson, A. and Almquist, Lena and Björkman, Patrik and Cesta, Amedeo and Cortellessa, Gabriella and Galindo, Cipriano and Gonzalez-Jimenez, Javier and Gustafsson, Kalle and Kiselev, Andrey and Loufti, Amy and Melendez-Fernandez, Francisco and Nilsson, Malin and Odens Hedman, Lasse and Odontidou, Eleni and Ruiz-Sarmiento, J. R. and Scherlund, Marten and Tiberio, L. and Rump, Stephen Von and Coradeschi, Silvia},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-KBS-Building_Multiversal_Semantic_Maps.pdf},
doi = {10.1162/PRES_a_00262},
year = {2017},
date = {2017-01-01},
journal = {Presence: Teleoperators and Virtual Environments},
abstract = {This paper reports on the EU project ExCITE with specific focus on the technical development of the telepresence platform over a period of 42 months. The aim of the project was to assess the robustness and validity of the mobile robotic telepresence (MRP) system Giraff as a means to support elderly and to foster their social interaction and participation. Embracing the idea of user-centered product refinement, the robot was tested over long periods of time in real homes. As such, the system development was driven by a strong involvement of elderly and their caregivers but also by technical challenges associated with deploying the robot in real-world contexts. The results of the 42-months-long evaluation is a system suitable for use in homes rather than a generic system suitable, for example, in office environments.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper reports on the EU project ExCITE with specific focus on the technical development of the telepresence platform over a period of 42 months. The aim of the project was to assess the robustness and validity of the mobile robotic telepresence (MRP) system Giraff as a means to support elderly and to foster their social interaction and participation. Embracing the idea of user-centered product refinement, the robot was tested over long periods of time in real homes. As such, the system development was driven by a strong involvement of elderly and their caregivers but also by technical challenges associated with deploying the robot in real-world contexts. The results of the 42-months-long evaluation is a system suitable for use in homes rather than a generic system suitable, for example, in office environments.
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Robot@Home, a robotic dataset for semantic mapping of home environments
The International Journal of Robotics Research, vol. 36, no. 2, pp. 131-141, 2017. Q1 · T1
@article{raul2017ijrr,
title = {Robot@Home, a robotic dataset for semantic mapping of home environments},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-IJRR-Robot_at_home_dataset.pdf
http://journals.sagepub.com/doi/abs/10.1177/0278364917695640},
doi = {10.1177/0278364917695640},
year = {2017},
date = {2017-01-01},
journal = {The International Journal of Robotics Research},
volume = {36},
number = {2},
pages = {131-141},
abstract = {This paper presents the Robot-at-Home dataset (Robot@Home), a collection of raw and processed sensory data from domestic settings aimed at serving as a benchmark for semantic mapping algorithms through the categorization of objects and/or rooms. The dataset contains 87,000+ time-stamped observations gathered by a mobile robot endowed with a rig of four RGB-D cameras and a 2D laser scanner. Raw observations have been processed to produce different outcomes also distributed with the dataset, including 3D reconstructions and 2D geometric maps of the inspected rooms, both annotated with the ground truth categories of the surveyed rooms and objects. The proposed dataset is particularly suited as a testbed for object and/or room categorization systems, but it can be also exploited for a variety of tasks, including robot localization, 3D map building, SLAM, and object segmentation. Robot@Home is publicly available for the research community at http://mapir.isa.uma.es/work/robot-at-home-dataset.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents the Robot-at-Home dataset (Robot@Home), a collection of raw and processed sensory data from domestic settings aimed at serving as a benchmark for semantic mapping algorithms through the categorization of objects and/or rooms. The dataset contains 87,000+ time-stamped observations gathered by a mobile robot endowed with a rig of four RGB-D cameras and a 2D laser scanner. Raw observations have been processed to produce different outcomes also distributed with the dataset, including 3D reconstructions and 2D geometric maps of the inspected rooms, both annotated with the ground truth categories of the surveyed rooms and objects. The proposed dataset is particularly suited as a testbed for object and/or room categorization systems, but it can be also exploited for a variety of tasks, including robot localization, 3D map building, SLAM, and object segmentation. Robot@Home is publicly available for the research community at http://mapir.isa.uma.es/work/robot-at-home-dataset.
2016
J. Gonzalez-Jimenez, V. Arevalo-Espejo, C. Galindo, J.R. Ruiz-Sarmiento
An Automated Surveying and Marking System for Continuous Setting-out of Tunnels
Computer-Aided Civil and Infrastructure Engineering, vol. 31, no. 3, pp. 219–228, 2016. Q1 · T1
@article{gonzalezjimenez2016,
title = {An Automated Surveying and Marking System for Continuous Setting-out of Tunnels},
author = {Gonzalez-Jimenez, Javier and Arevalo-Espejo, Vicente and Galindo, Cipriano and Ruiz-Sarmiento, J. R.},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016-jgonzalez-CACIE-TCS.pdf},
doi = {10.1111/mice.12158},
year = {2016},
date = {2016-03-01},
journal = {Computer-Aided Civil and Infrastructure Engineering},
volume = {31},
number = {3},
pages = {219--228},
abstract = {Setting-out is a crucial and frequently repeated process in construction and civil engineering. It is carried out by qualified operators who, making use of surveying stations, identify reference points to guide workers in their tasks. In this paper, we focus on the particular case of setting-out road/train tunnels and propose a system that automatically performs the setting-out operation of the tunnel section to be perforated. The presented system, called Tunnel Continuous Setout (TCS), integrates a scanning device that surveys the excavation front and a laser projector that continuously displays the actual tunnel section computed as the intersection between the surveyed terrain and the planned tunnel section. Thus, the topographer intervention is only required for the precise positioning of the TCS device at the beginning of each working stretch, which depends on the operational range of its components (limited to 25 meters in the current implementation). A prototype of the TCS system has been employed in a real construction site proving its benefits and advantages with respect to the traditional setting-out techniques.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Setting-out is a crucial and frequently repeated process in construction and civil engineering. It is carried out by qualified operators who, making use of surveying stations, identify reference points to guide workers in their tasks. In this paper, we focus on the particular case of setting-out road/train tunnels and propose a system that automatically performs the setting-out operation of the tunnel section to be perforated. The presented system, called Tunnel Continuous Setout (TCS), integrates a scanning device that surveys the excavation front and a laser projector that continuously displays the actual tunnel section computed as the intersection between the surveyed terrain and the planned tunnel section. Thus, the topographer intervention is only required for the precise positioning of the TCS device at the beginning of each working stretch, which depends on the operational range of its components (limited to 25 meters in the current implementation). A prototype of the TCS system has been employed in a real construction site proving its benefits and advantages with respect to the traditional setting-out techniques.
2015
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Exploiting Semantic Knowledge for Robot Object Recognition
Knowledge-Based Systems, vol. 86, pp. 131–142, 2015. Q1 · T1
@article{ruizsarmiento2015,
title = {Exploiting Semantic Knowledge for Robot Object Recognition},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2015/2015-raul-KBS-Exploiting_Semantic_Knowledge_for_Robot_Object_Recognition.pdf},
doi = {doi:10.1016/j.knosys.2015.05.032},
year = {2015},
date = {2015-01-01},
journal = {Knowledge-Based Systems},
volume = {86},
pages = {131--142},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Expert Systems with Applications, vol. 42, no. 22, pp. 8805–8816, 2015. Q1 · T1
@article{ruizsarmiento2015e,
title = {Scene Object Recognition for Mobile Robots Through Semantic Knowledge and Probabilistic Graphical Models},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2015/2015-raul-ESWA-Scene_Object_Recognition_for_Mobile_Robots_Through_Semantic_Knowledge_and_Probabilistic_Graphical_Models.pdf},
doi = {doi:10.1016/j.eswa.2015.07.033},
year = {2015},
date = {2015-01-01},
journal = {Expert Systems with Applications},
volume = {42},
number = {22},
pages = {8805--8816},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Book Chapters (7)
2021
A.M. Burgueño, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Autonomous Docking of Mobile Robots by Reinforcement Learning Tackling the Sparse Reward Problem
vol. 12862, Chapter Advances in Computational Intelligence, pp. 392-403, Springer International Publishing, Cham, 2021, ISBN: 978-3-030-85099-9.
@inbook{burgeno2021docking,
title = {Autonomous Docking of Mobile Robots by Reinforcement Learning Tackling the Sparse Reward Problem},
author = {Burgueño, A. M. and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2021/2021_burgeno_IWAN_Autonomous_docking_of_mobile_robots.pdf},
doi = {10.1007/978-3-030-85099-9_32},
isbn = {978-3-030-85099-9},
year = {2021},
date = {2021-01-01},
volume = {12862},
pages = {392-403},
publisher = {Springer International Publishing},
address = {Cham},
chapter = {Advances in Computational Intelligence},
series = {Lecture Notes in Computer Science},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2019
D. Fernandez-Chaves, J.R. Ruiz-Sarmiento, N. Petkov, J. Gonzalez-Jimenez
Integration of CNN into a Robotic Architecture to Build Semantic Maps of Indoor Environments
vol. 11507, pp. 313–324, Springer, Cham, 2019, ISBN: 978-3-030-20517-1.
@inbook{davfercha_IWANN_2019_integration,
title = {Integration of CNN into a Robotic Architecture to Build Semantic Maps of Indoor Environments},
author = {Fernandez-Chaves, David and Ruiz-Sarmiento, J. R. and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://drive.google.com/uc?id=1ed-sxK8OPM7fG1rwqp_y0GGKxBC8DktF&export=download&authuser=0},
doi = {10.1007/978-3-030-20518-8_27},
isbn = {978-3-030-20517-1},
year = {2019},
date = {2019-05-01},
volume = {11507},
pages = {313--324},
publisher = {Springer, Cham},
series = {Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science},
abstract = {In robotics, semantic mapping refers to the construction ofa rich representation of the environment that includes high level infor-mation needed by the robot to accomplish its tasks. Building a semanticmap requires algorithms to process sensor data at different levels: geo-metric, topological and object detections/categories, which must be inte-grated into an unified model. This paper describes a robotic architecturethat successfully builds such semantic maps for indoor environments. Forthis purpose, within a ROS-based ecosystem, we apply a state-of-the-artConvolutional Neural Network (CNN), concretely YOLOv3, for detect-ing objects in images. The detection results are placed within a geometricmap of the environment making use of a number of modules of the ar-chitecture: robot localization, camera extrinsic calibration, data form adepth camera, etc. We demonstrate the suitability of the proposed frame-work by building semantic maps of several home environments from theRobot@Home dataset, using Unity 3D as a tool to visualize the maps aswell as to provide future robotic developments.},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
In robotics, semantic mapping refers to the construction ofa rich representation of the environment that includes high level infor-mation needed by the robot to accomplish its tasks. Building a semanticmap requires algorithms to process sensor data at different levels: geo-metric, topological and object detections/categories, which must be inte-grated into an unified model. This paper describes a robotic architecturethat successfully builds such semantic maps for indoor environments. Forthis purpose, within a ROS-based ecosystem, we apply a state-of-the-artConvolutional Neural Network (CNN), concretely YOLOv3, for detect-ing objects in images. The detection results are placed within a geometricmap of the environment making use of a number of modules of the ar-chitecture: robot localization, camera extrinsic calibration, data form adepth camera, etc. We demonstrate the suitability of the proposed frame-work by building semantic maps of several home environments from theRobot@Home dataset, using Unity 3D as a tool to visualize the maps aswell as to provide future robotic developments.
D. Zuñiga-Noël, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Intrinsic Calibration of Depth Cameras for Mobile Robots using a Radial Laser Scanner
vol. 11678, pp. 659-671, Springer International Publishing, 2019, ISBN: 978-3-030-29888-3.
@inbook{zuniga2019caip,
title = {Intrinsic Calibration of Depth Cameras for Mobile Robots using a Radial Laser Scanner},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.1007/978-3-030-29888-3_54},
isbn = {978-3-030-29888-3},
year = {2019},
date = {2019-01-01},
volume = {11678},
pages = {659-671},
publisher = {Springer International Publishing},
series = {Computer Analysis of Images and Patterns. Lecture Notes in Computer Science},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2018
J.R. Ruiz-Sarmiento, J. Monroy, F.A. Moreno, J.M. Bonelo, J. Gonzalez-Jimenez
Frontiers in Artificial Intelligence and Applications (FAIA), IOS Press, pp. 122 - 133, IOS Press, 2018.
@inbook{ruizsarmiento_faia18,
title = {Analysis of Data from the Industrial Machinery within the Hot Rolling Process for Predictive Maintenance},
author = {Ruiz-Sarmiento, J. R. and Monroy, Javier and Moreno, Francisco-Angel and Bonelo, J. M. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018-raul-FAIA-Analysis_of_Data_from_the_Industrial_Machinery_within_the_Hot_Rolling_Process.pdf},
doi = {10.3233/978-1-61499-929-4-122},
year = {2018},
date = {2018-01-01},
booktitle = {Frontiers in Artificial Intelligence and Applications (FAIA), IOS Press},
pages = {122 - 133},
publisher = {IOS Press},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
J. Monroy, J.R. Ruiz-Sarmiento, F.A. Moreno, C. Galindo, J. Gonzalez-Jimenez
Towards a Semantic Gas Source Localization under Uncertainty
Information Processing and Management of Uncertainty in Knowledge-Based Systems. Applications, pp. 504–516, Springer, Cham, 2018, ISBN: 978-3-319-91479-4.
@inbook{monroy_2018_ipmu,
title = {Towards a Semantic Gas Source Localization under Uncertainty},
author = {Monroy, Javier and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_monroy_IPMU_Semantic_Gas_Source_Localization.pdf},
doi = {10.1007/978-3-319-91479-4_42},
isbn = {978-3-319-91479-4},
year = {2018},
date = {2018-01-01},
booktitle = {Information Processing and Management of Uncertainty in Knowledge-Based Systems. Applications},
pages = {504--516},
publisher = {Springer, Cham},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2016
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
10th International Conference on Ubiquitous Computing and Ambient Intelligence, vol. 10070, Springer, 2016, ISBN: 978-3-319-48798-4.
@inbook{ruizsarmiento2016,
title = {Probability and Common-Sense: Tandem Towards Robust Robotic Object Recognition in Ambient Assisted Living},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016-raul-UCAmI-Probability_and_Common-Sense_Tandem.pdf},
doi = {10.1007/978-3-319-48799-1_1},
isbn = {978-3-319-48798-4},
year = {2016},
date = {2016-01-01},
booktitle = {10th International Conference on Ubiquitous Computing and Ambient Intelligence},
volume = {10070},
publisher = {Springer},
series = {Lecture Notes in Computer Science},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2011
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Improving Human Face Detection through TOF Cameras for Ambient Intelligence Applications
International Symposium on Ambient Intelligence, Salamanca, 2011.
@inbook{ruizsarmiento2011,
title = {Improving Human Face Detection through TOF Cameras for Ambient Intelligence Applications},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/cgalindo/papers/draft-ISAMI11.pdf},
doi = {10.1007/978-3-642-19937-0_16},
year = {2011},
date = {2011-04-01},
booktitle = {International Symposium on Ambient Intelligence},
address = {Salamanca},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
Conferences (27)
2025
J. Moncada-Ramirez, J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Mobile Robot Place Segmentation and Categorization Using Object Semantics
European Conference on Mobile Robots, 2025.
@inproceedings{moncada2025ecmr,
title = {Mobile Robot Place Segmentation and Categorization Using Object Semantics},
author = {Moncada-Ramirez, Jesús and Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://ieeexplore.ieee.org/document/11163204},
doi = {10.1109/ECMR65884.2025.11163204},
year = {2025},
date = {2025-09-01},
booktitle = {European Conference on Mobile Robots},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2024
A. Galeote-Luque, V. Kubelka, M. Magnusson, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Doppler-only Single-scan 3D Vehicle Odometry
2024.
@inproceedings{galeote2024icra,
title = {Doppler-only Single-scan 3D Vehicle Odometry},
author = {Galeote-Luque, Andres and Kubelka, Vladimír and Magnusson, Martin and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://arxiv.org/abs/2310.04113},
year = {2024},
date = {2024-05-01},
journal = {IEEE International Conference on Robotics and Automation},
abstract = {We present a novel 3D odometry method that recovers the full motion of a vehicle only from a Doppler-capable range sensor. It leverages the radial velocities measured from the scene, estimating the sensor's velocity from a single scan. The vehicle's 3D motion, defined by its linear and angular velocities, is calculated taking into consideration its kinematic model which provides a constraint between the velocity measured at the sensor frame and the vehicle frame.
Experiments carried out prove the viability of our single-sensor method compared to mounting an additional IMU. Our method provides the translation of the sensor, which cannot be reliably determined from an IMU, as well as its rotation. Its short-term accuracy and fast operation (~5ms) make it a proper candidate to supply the initialization to more complex localization algorithms or mapping pipelines. Not only does it reduce the error of the mapper, but it does so at a comparable level of accuracy as an IMU would. All without the need to mount and calibrate an extra sensor on the vehicle.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a novel 3D odometry method that recovers the full motion of a vehicle only from a Doppler-capable range sensor. It leverages the radial velocities measured from the scene, estimating the sensor's velocity from a single scan. The vehicle's 3D motion, defined by its linear and angular velocities, is calculated taking into consideration its kinematic model which provides a constraint between the velocity measured at the sensor frame and the vehicle frame.
Experiments carried out prove the viability of our single-sensor method compared to mounting an additional IMU. Our method provides the translation of the sensor, which cannot be reliably determined from an IMU, as well as its rotation. Its short-term accuracy and fast operation (~5ms) make it a proper candidate to supply the initialization to more complex localization algorithms or mapping pipelines. Not only does it reduce the error of the mapper, but it does so at a comparable level of accuracy as an IMU would. All without the need to mount and calibrate an extra sensor on the vehicle.
Experiments carried out prove the viability of our single-sensor method compared to mounting an additional IMU. Our method provides the translation of the sensor, which cannot be reliably determined from an IMU, as well as its rotation. Its short-term accuracy and fast operation (~5ms) make it a proper candidate to supply the initialization to more complex localization algorithms or mapping pipelines. Not only does it reduce the error of the mapper, but it does so at a comparable level of accuracy as an IMU would. All without the need to mount and calibrate an extra sensor on the vehicle.
2023
A.J. Pérez Bazuelo, J.R. Ruiz-Sarmiento, G. Ambrosio, J. Gonzalez-Jimenez
Towards a Voxelized Semantic Representation of the Workspace of Mobile Robots
Advances in Computational Intelligence. IWANN, pp. 194–205, Springer Nature, Switzerland, 2023, ISBN: 978-3-031-43078-7.
@inproceedings{raul2023iwann,
title = {Towards a Voxelized Semantic Representation of the Workspace of Mobile Robots},
author = {Pérez Bazuelo, Antonio Jesús and Ruiz-Sarmiento, J. R. and Ambrosio, Gregorio and Gonzalez-Jimenez, Javier},
doi = {10.1007/978-3-031-43078-7_16},
isbn = {978-3-031-43078-7},
year = {2023},
date = {2023-01-01},
booktitle = {Advances in Computational Intelligence. IWANN},
volume = {14135},
pages = {194--205},
publisher = {Springer Nature},
address = {Switzerland},
series = {Lecture Notes in Computer Science},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2022
P. Ojeda, J.R. Ruiz-Sarmiento, J. Monroy, J. Gonzalez-Jimenez
Fifth Iberian Robotics Conference (ROBOT2022), Zaragoza, Spain, 2022.
@inproceedings{ojeda_robot2022_gadentools,
title = {GadenTools: A Toolkit for Testing and Simulating Robotic Olfaction Tasks With Jupyter Notebook Support},
author = {Ojeda, Pepe and Ruiz-Sarmiento, J. R. and Monroy, Javier and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2022/2022_ojeda_ROBOT2022_GadenTools.pdf},
doi = {10.1007/978-3-031-21062-4_14},
year = {2022},
date = {2022-01-01},
booktitle = {Fifth Iberian Robotics Conference (ROBOT2022)},
address = {Zaragoza, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2021
H. Howard-Jenkins, J.R. Ruiz-Sarmiento, V.A. Prisacariu
LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10107-10116, 2021.
@inproceedings{raul2021iccv,
title = {LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments},
author = {Howard-Jenkins, Henry and Ruiz-Sarmiento, J. R. and Prisacariu, Victor Adrian},
url = {https://openaccess.thecvf.com/content/ICCV2021/papers/Howard-Jenkins_LaLaLoc_Latent_Layout_Localisation_in_Dynamic_Unvisited_Environments_ICCV_2021_paper.pdf},
doi = {10.48550/arXiv.2104.09169},
year = {2021},
date = {2021-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
pages = {10107-10116},
abstract = {We present LaLaLoc to localise in environments without the need for prior visitation, and in a manner that is robust to large changes in scene appearance, such as a full rearrangement of furniture. Specifically, LaLaLoc performs localisation through latent representations of room layout. LaLaLoc learns a rich embedding space shared between RGB panoramas and layouts inferred from a known floor plan that encodes the structural similarity between locations. Further, LaLaLoc introduces direct, cross-modal pose optimisation in its latent space. Thus, LaLaLoc enables fine-grained pose estimation in a scene without the need for prior visitation, as well as being robust to dynamics, such as a change in furniture configuration. We show that in a domestic environment LaLaLoc is able to accurately localise a single RGB panorama image to within 8.3cm, given only a floor plan as a prior.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present LaLaLoc to localise in environments without the need for prior visitation, and in a manner that is robust to large changes in scene appearance, such as a full rearrangement of furniture. Specifically, LaLaLoc performs localisation through latent representations of room layout. LaLaLoc learns a rich embedding space shared between RGB panoramas and layouts inferred from a known floor plan that encodes the structural similarity between locations. Further, LaLaLoc introduces direct, cross-modal pose optimisation in its latent space. Thus, LaLaLoc enables fine-grained pose estimation in a scene without the need for prior visitation, as well as being robust to dynamics, such as a change in furniture configuration. We show that in a domestic environment LaLaLoc is able to accurately localise a single RGB panorama image to within 8.3cm, given only a floor plan as a prior.
D. Fernandez-Chaves, J.L. Matez-Bandera, J.R. Ruiz-Sarmiento, J. Monroy, N. Petkov, J. Gonzalez-Jimenez
Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics
International Conference on Computer Analysis of Images and Patterns (CAIP), 2021.
@inproceedings{fernandez_caip_2021,
title = {Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics},
author = {Fernandez-Chaves, David and Matez-Bandera, J. L and Ruiz-Sarmiento, J. R. and Monroy, Javier and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2021/2021_fernandezchaves_CAIP_espation_temporal_coherence_objects.pdf},
doi = {10.1007/978-3-030-89131-2_17},
year = {2021},
date = {2021-01-01},
booktitle = {International Conference on Computer Analysis of Images and Patterns (CAIP)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2020
A.M. Burgueño, J.R. Ruiz-Sarmiento, F.A. Moreno, J. Gonzalez-Jimenez
A Collection of Jupyter Notebooks Covering the Fundamentals of Computer Vision
ICERI2020 Proceedings, pp. 5495-5505, IATED, 2020, ISBN: 978-84-09-24232-0.
@inproceedings{burgeno2020iceri,
title = {A Collection of Jupyter Notebooks Covering the Fundamentals of Computer Vision},
author = {Burgueño, A. M. and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
url = {http://dx.doi.org/10.21125/iceri.2020.1189},
doi = {10.21125/iceri.2020.1189},
isbn = {978-84-09-24232-0},
year = {2020},
date = {2020-11-01},
booktitle = {ICERI2020 Proceedings},
pages = {5495-5505},
publisher = {IATED},
series = {13th annual International Conference of Education, Research and Innovation},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
S.F. Baltanas-Molero, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Empowering Mobile Robotics Undergraduate Courses by Using Jupyter Notebooks
14th annual International Technology, Education and Development Conference, Valencia, Spain, 2020.
@inproceedings{baltanas2019inted,
title = {Empowering Mobile Robotics Undergraduate Courses by Using Jupyter Notebooks},
author = {Baltanas-Molero, Samuel-Felipe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
year = {2020},
date = {2020-03-01},
booktitle = {14th annual International Technology, Education and Development Conference},
address = {Valencia, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
S.F. Baltanas-Molero, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
A Face Recognition System for Assistive Robots
International Conference on Applications of Intelligent Systems (APPIS), Las Palmas de Gran Canaria (Spain), 2020.
@inproceedings{sambalmol_APPIS_2020_face,
title = {A Face Recognition System for Assistive Robots},
author = {Baltanas-Molero, Samuel-Felipe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.1145/3378184.3378225},
year = {2020},
date = {2020-01-01},
booktitle = {International Conference on Applications of Intelligent Systems (APPIS)},
address = {Las Palmas de Gran Canaria (Spain)},
abstract = {Assistive robots collaborating with people demand strong Human-Robot interaction capabilities. In this way, recognizing the person the robot has to interact with is paramount to provide a personalized service and reach a satisfactory end-user experience. To this end, face recognition: a non-intrusive, automatic mechanism of identification using biometric identifiers from an user’s face, has gained relevance in the recent years, as the advances in machine learning and the creation of huge public datasets have considerably improved the state-of-the-art performance. In this work we study different open-source implementations of the typical components of state-of-the-art face recognition pipelines, including face detection, feature extraction and classification, and propose a recognition system integrating the most suitable methods for their utilization in assistant robots. Concretely, for face detection we have considered MTCNN, OpenCV’s DNN, and OpenPose, while for feature extraction we have analyzed InsightFace and Facenet. We have made public an implementation of the proposed recognition framework, ready to be used by any robot running the Robot Operating System (ROS). The methods in the spotlight have been compared in terms of accuracy and performance in common benchmark datasets, namely FDDB and LFW, to aid the choice of the final system implementation, which has been tested in a real robotic platform.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Assistive robots collaborating with people demand strong Human-Robot interaction capabilities. In this way, recognizing the person the robot has to interact with is paramount to provide a personalized service and reach a satisfactory end-user experience. To this end, face recognition: a non-intrusive, automatic mechanism of identification using biometric identifiers from an user’s face, has gained relevance in the recent years, as the advances in machine learning and the creation of huge public datasets have considerably improved the state-of-the-art performance. In this work we study different open-source implementations of the typical components of state-of-the-art face recognition pipelines, including face detection, feature extraction and classification, and propose a recognition system integrating the most suitable methods for their utilization in assistant robots. Concretely, for face detection we have considered MTCNN, OpenCV’s DNN, and OpenPose, while for feature extraction we have analyzed InsightFace and Facenet. We have made public an implementation of the proposed recognition framework, ready to be used by any robot running the Robot Operating System (ROS). The methods in the spotlight have been compared in terms of accuracy and performance in common benchmark datasets, namely FDDB and LFW, to aid the choice of the final system implementation, which has been tested in a real robotic platform.
D. Fernandez-Chaves, J.R. Ruiz-Sarmiento, N. Petkov, J. Gonzalez-Jimenez
From Object Detection to Room Categorization in Robotics
Las Palmas de Gran Canaria (Spain), 2020.
@inproceedings{davfercha_APPIS_2020_room_categorization,
title = {From Object Detection to Room Categorization in Robotics},
author = {Fernandez-Chaves, David and Ruiz-Sarmiento, J. R. and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {https://drive.google.com/uc?id=1pUd5L1to0M4TkzPlISQzXgV2z8OaEu89&export=download&authuser=0},
doi = {10.1145/3378184.3378230},
year = {2020},
date = {2020-01-01},
journal = {International Conference on Applications of Intelligent Systems (APPIS)},
address = {Las Palmas de Gran Canaria (Spain)},
abstract = {This article deals with the problem of room categorization, i.e. the classification of a room as being a bathroom, kitchen, living-room, bedroom, etc., by an autonomous robot operating in home environments. For that, we propose a room categorization system based on a Bayesian probabilistic framework that combines object detections and its semantics. For detecting objects we resort to a state-of-the-art CNN, Mask R-CNN, while the meaning or semantics of those detections is provided by an ontology. Such an ontology encodes the relations between object and room categories, that is, in which room types the different object categories are typically found (toilets in bathrooms, microwaves in kitchens, etc.). The Bayesian framework is in charge of fusing both sources of information and providing a probability distribution over the set of categories the room can belong to. The proposed system has been evaluated in houses from the Robot@Home dataset, validating its effectiveness under real-world conditions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This article deals with the problem of room categorization, i.e. the classification of a room as being a bathroom, kitchen, living-room, bedroom, etc., by an autonomous robot operating in home environments. For that, we propose a room categorization system based on a Bayesian probabilistic framework that combines object detections and its semantics. For detecting objects we resort to a state-of-the-art CNN, Mask R-CNN, while the meaning or semantics of those detections is provided by an ontology. Such an ontology encodes the relations between object and room categories, that is, in which room types the different object categories are typically found (toilets in bathrooms, microwaves in kitchens, etc.). The Bayesian framework is in charge of fusing both sources of information and providing a probability distribution over the set of categories the room can belong to. The proposed system has been evaluated in houses from the Robot@Home dataset, validating its effectiveness under real-world conditions.
2019
M. Luperto, J. Monroy, J.R. Ruiz-Sarmiento, F.A. Moreno, N. Basilico, J. Gonzalez-Jimenez, N.A. Borghese
Towards Long-Term Deployment of a Mobile Robot for at-Home Ambient Assisted Living of the Elderly
European Conference on Mobile Robots (ECMR), Prague, Czech Republic, 2019.
@inproceedings{luperto_ecmr_19,
title = {Towards Long-Term Deployment of a Mobile Robot for at-Home Ambient Assisted Living of the Elderly},
author = {Luperto, Matteo and Monroy, Javier and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Basilico, Nicola and Gonzalez-Jimenez, Javier and Borghese, N. Alberto},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019_luperto_ECMR_Long_term_deployment_AAS.pdf},
doi = {10.1109/ECMR.2019.8870924},
year = {2019},
date = {2019-09-01},
booktitle = {European Conference on Mobile Robots (ECMR)},
address = {Prague, Czech Republic},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J. Monroy, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
An Evaluation of Plume Tracking as a Strategy for Gas Source Localization in Turbulent Wind Flows
ISOCS/IEEE International Symposium on Olfaction and Electronic Nose (ISOEN), Fukuoka (Japan), 2019.
@inproceedings{monroy_ISOEN_2019_evaluation,
title = {An Evaluation of Plume Tracking as a Strategy for Gas Source Localization in Turbulent Wind Flows},
author = {Monroy, Javier and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019_monroy_ISOEN_Plume_Tracking_Evaluation.pdf},
doi = {10.1109/ISOEN.2019.8823147},
year = {2019},
date = {2019-05-01},
booktitle = {ISOCS/IEEE International Symposium on Olfaction and Electronic Nose (ISOEN)},
address = {Fukuoka (Japan)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J.R. Ruiz-Sarmiento, J. Monroy, F.A. Moreno, J. Gonzalez-Jimenez
A Tutorial on Object Recognition by Machine Learning Techniques Using Python
INTED2019 Proceedings, pp. 3321-3330, IATED, Valencia, SPAIN, 2019, ISBN: 978-84-09-08619-1.
@inproceedings{ruiz2019inted,
title = {A Tutorial on Object Recognition by Machine Learning Techniques Using Python},
author = {Ruiz-Sarmiento, J. R. and Monroy, Javier and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019-raul-INTED-A_Tutorial_on_Object_recognition_by_Machine_Learning_Techniques_Using_Python.pdf},
doi = {10.21125/inted.2019.0864},
isbn = {978-84-09-08619-1},
year = {2019},
date = {2019-03-01},
booktitle = {INTED2019 Proceedings},
pages = {3321-3330},
publisher = {IATED},
address = {Valencia, SPAIN},
series = {13th International Technology, Education and Development Conference},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J. Monroy, J.R. Ruiz-Sarmiento, F.A. Moreno, C. Galindo, J. Gonzalez-Jimenez
Improvement of the Sensory and Autonomous Capability of Robots Through Olfaction: the IRO Project
Spanish Robotics Conference, Alicante (Spain), 2019.
@inproceedings{monroy_JNR_2019,
title = {Improvement of the Sensory and Autonomous Capability of Robots Through Olfaction: the IRO Project},
author = {Monroy, Javier and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2019/2019_monroy_JNR_The_IRO_Project.pdf},
year = {2019},
date = {2019-01-01},
booktitle = {Spanish Robotics Conference},
address = {Alicante (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2018
J.R. Ruiz-Sarmiento, F.A. Moreno, J. Monroy, J. Gonzalez-Jimenez
mVision, a Toolbox for Computer Vision Courses
12th annual International Technology, Education and Development Conference (INTED'18), pp. 5232–5241, Valencia (Spain), 2018, ISBN: 978-84-697-9480-7.
@inproceedings{raul2018mvision,
title = {mVision, a Toolbox for Computer Vision Courses},
author = {Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Monroy, Javier and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018-raul-INTED-mVision.pdf},
doi = {10.21125/inted.2018.1231},
isbn = {978-84-697-9480-7},
year = {2018},
date = {2018-03-01},
booktitle = {12th annual International Technology, Education and Development Conference (INTED'18)},
pages = {5232--5241},
address = {Valencia (Spain)},
abstract = {This paper presents the mVision toolbox to the lecturers’ community and computer vision practitioners, which is publicly available at https://github.com/jotaraul/mVision. mVision provides a number of intuitive Graphical User Interfaces (GUIs) to apply different computer vision techniques like: image smoothing or enhancement, edge detection, segmentation into regions, etc. The interesting point of these GUIs is that they offer handy ways to select and configure different popular algorithms and, in addition to the results of their execution (e.g. detected edges or regions), they also provide intermediate information that is useful for understanding what it is underneath. With these features, the toolbox attempts to fill the gap between low level implementations of computer vision techniques, which are efficient resources but hard to leverage for teaching purposes, and high level interfaces that bring to you just the output of these techniques, neglecting interesting intermediate results. In this way, mVision aims to become an engaging tool for both lecturers, pursuing a better and quicker understanding of the basis of these techniques, and computer vision students and practitioners seeking to enhance that basis. The toolbox is written for MATLAB, a popular software application in academia. This work also yields directions about its utilization for educational purposes, and describes our experience with it in undergraduate and graduate courses since 2009.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents the mVision toolbox to the lecturers’ community and computer vision practitioners, which is publicly available at https://github.com/jotaraul/mVision. mVision provides a number of intuitive Graphical User Interfaces (GUIs) to apply different computer vision techniques like: image smoothing or enhancement, edge detection, segmentation into regions, etc. The interesting point of these GUIs is that they offer handy ways to select and configure different popular algorithms and, in addition to the results of their execution (e.g. detected edges or regions), they also provide intermediate information that is useful for understanding what it is underneath. With these features, the toolbox attempts to fill the gap between low level implementations of computer vision techniques, which are efficient resources but hard to leverage for teaching purposes, and high level interfaces that bring to you just the output of these techniques, neglecting interesting intermediate results. In this way, mVision aims to become an engaging tool for both lecturers, pursuing a better and quicker understanding of the basis of these techniques, and computer vision students and practitioners seeking to enhance that basis. The toolbox is written for MATLAB, a popular software application in academia. This work also yields directions about its utilization for educational purposes, and describes our experience with it in undergraduate and graduate courses since 2009.
M. Luperto, J. Monroy, F.A. Moreno, J.R. Ruiz-Sarmiento, N. Basilico, J. Gonzalez-Jimenez, N.A. Borghese
A Multi-Actor Framework Centered around an Assististe Mobile Robot for Elderly People Living Alone
IEEE International Conference on Intelligent Robots - Workshop on Robots for Assisted Living (IROS), 2018.
@inproceedings{luperto_IROS2018,
title = {A Multi-Actor Framework Centered around an Assististe Mobile Robot for Elderly People Living Alone},
author = {Luperto, Matteo and Monroy, Javier and Moreno, Francisco-Angel and Ruiz-Sarmiento, J. R. and Basilico, Nicola and Gonzalez-Jimenez, Javier and Borghese, N. Alberto},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_luperto_IROS__Assisted_Living_of_Pre_Frail_Elders_Using_an_Autonomous_Robot_and_a_Multi_Actor_Framework.pdf},
year = {2018},
date = {2018-01-01},
booktitle = {IEEE International Conference on Intelligent Robots - Workshop on Robots for Assisted Living (IROS)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
F.A. Moreno, J.R. Ruiz-Sarmiento, J. Monroy, M. Fernandez, J. Gonzalez-Jimenez
Analyzing Interference Between RGB-D Cameras for Human Motion Tracking
International Conference on Applications of Intelligent Systems (APPIS), Las Palmas de Gran Canaria (Spain), 2018.
@inproceedings{moreno2018appis,
title = {Analyzing Interference Between RGB-D Cameras for Human Motion Tracking},
author = {Moreno, Francisco-Angel and Ruiz-Sarmiento, J. R. and Monroy, Javier and Fernandez, Maïlys and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_moreno_APPIS Analyzing interference between RGBD cameras.pdf},
year = {2018},
date = {2018-01-01},
booktitle = {International Conference on Applications of Intelligent Systems (APPIS)},
address = {Las Palmas de Gran Canaria (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J.R. Ruiz-Sarmiento, J. Monroy, F.A. Moreno, J.M. Bonelo, J. Gonzalez-Jimenez
Data Analysis of the Hot Rolling Process in a Stainless Steel Factory for Predictive Maintenance
International Conference on Applications of Intelligent Systems (APPIS), Las Palmas de Gran Canaria (Spain), 2018.
@inproceedings{ruizsarmiento2018,
title = {Data Analysis of the Hot Rolling Process in a Stainless Steel Factory for Predictive Maintenance},
author = {Ruiz-Sarmiento, J. R. and Monroy, Javier and Moreno, Francisco-Angel and Bonelo, J. M. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_raul_APPIS_Data Analysis of the Hot Rolling Process.pdf},
year = {2018},
date = {2018-01-01},
booktitle = {International Conference on Applications of Intelligent Systems (APPIS)},
address = {Las Palmas de Gran Canaria (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2017
J.R. Ruiz-Sarmiento, M. Guenther, C. Galindo, J. Gonzalez-Jimenez, J. Hertzberg
Online Context-based Object Recognition for Mobile Robots
17th International Conference on Autonomous Robot Systems and Competition (ICARSC), Coimbra, Portugal, 2017.
@inproceedings{raul2017icarsc,
title = {Online Context-based Object Recognition for Mobile Robots},
author = {Ruiz-Sarmiento, J. R. and Guenther, Martin and Galindo, Cipriano and Gonzalez-Jimenez, Javier and Hertzberg, Joachim},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-ICARSC-Online_Context-based_Object_Recognition_for_Mobile_Robots.pdf},
year = {2017},
date = {2017-04-01},
booktitle = {17th International Conference on Autonomous Robot Systems and Competition (ICARSC)},
address = {Coimbra, Portugal},
abstract = {This work proposes a robotic object recognition system that takes advantage of the contextual information latent in human-like environments in an online fashion. To fully leverage context, it is needed perceptual information from (at least) a portion of the scene containing the objects of interest, which could not be entirely covered by just an one-shot sensor observation. Information from a larger portion of the scenario could still be considered by progressively registering observations, but this approach experiences difficulties under some circumstances, e.g. limited and heavily demanded computational resources, dynamic environments, etc. Instead of this, the proposed recognition system relies on an anchoring process for the fast registration and propagation of objects' features and locations beyond the current sensor frustum. In this way, the system builds a graph-based world model containing the objects in the scenario (both in the current and previously perceived shots), which is exploited by a Probabilistic Graphical Model (PGM) in order to leverage contextual information during recognition. We also propose a novel way to include the outcome of local object recognition methods in the PGM, which results in a decrease in the usually high CRF learning complexity. A demonstration of our proposal has been conducted employing a dataset captured by a mobile robot from restaurant-like settings, showing promising results.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work proposes a robotic object recognition system that takes advantage of the contextual information latent in human-like environments in an online fashion. To fully leverage context, it is needed perceptual information from (at least) a portion of the scene containing the objects of interest, which could not be entirely covered by just an one-shot sensor observation. Information from a larger portion of the scenario could still be considered by progressively registering observations, but this approach experiences difficulties under some circumstances, e.g. limited and heavily demanded computational resources, dynamic environments, etc. Instead of this, the proposed recognition system relies on an anchoring process for the fast registration and propagation of objects' features and locations beyond the current sensor frustum. In this way, the system builds a graph-based world model containing the objects in the scenario (both in the current and previously perceived shots), which is exploited by a Probabilistic Graphical Model (PGM) in order to leverage contextual information during recognition. We also propose a novel way to include the outcome of local object recognition methods in the PGM, which results in a decrease in the usually high CRF learning complexity. A demonstration of our proposal has been conducted employing a dataset captured by a mobile robot from restaurant-like settings, showing promising results.
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Experiences on a Motivational Learning Approach for Robotics in Undergraduate Courses
INTED2017 Proceedings, pp. 3803-3811, IATED, Valencia, Spain, 2017, ISBN: 978-84-617-8491-2.
@inproceedings{ruiz-sarmiento2017inted,
title = {Experiences on a Motivational Learning Approach for Robotics in Undergraduate Courses},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-INTED-Experiences_on_a_Motivational_Learning_Approach_for_Robotics.pdf},
isbn = {978-84-617-8491-2},
year = {2017},
date = {2017-03-01},
booktitle = {INTED2017 Proceedings},
pages = {3803-3811},
publisher = {IATED},
address = {Valencia, Spain},
series = {11th annual International Technology, Education and Development Conference},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2015
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Joint Categorization of Objects and Rooms for Mobile Robots
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015.
@inproceedings{ruizsarmiento2015d,
title = {Joint Categorization of Objects and Rooms for Mobile Robots},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2015/2015-raul-IROS-Joint_Categorization_of_Objects_and_Rooms_for_Mobile_Robots.pdf},
year = {2015},
date = {2015-01-01},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
OLT: A Toolkit for Object Labeling Applied to Robotic RGB-D Datasets
European Conference on Mobile Robots, Lincoln, UK, 2015.
@inproceedings{ruizsarmiento2015c,
title = {OLT: A Toolkit for Object Labeling Applied to Robotic RGB-D Datasets},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2015/2015-raul-ECMR-OLT.pdf},
year = {2015},
date = {2015-01-01},
booktitle = {European Conference on Mobile Robots},
address = {Lincoln, UK},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
UPGMpp: a Software Library for Contextual Object Recognition
3rd. Workshop on Recognition and Action for Scene Understanding (REACTS), 2015.
@inproceedings{ruizsarmiento2015b,
title = {UPGMpp: a Software Library for Contextual Object Recognition},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2015/2015-raul-REACTS-UPGMpp.pdf},
year = {2015},
date = {2015-01-01},
booktitle = {3rd. Workshop on Recognition and Action for Scene Understanding (REACTS)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2014
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
European Conf. on Artificial Intelligence. Workshop on Cognitive Robotics, 2014.
@inproceedings{ruizsarmiento2014,
title = {Mobile Robot Object Recognition through the Synergy of Probabilistic Graphical Models and Semantic Knowledge},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {http://www.ida.liu.se/conferences/cogrob2014/papers/cogrob2014_submission_9.pdf},
year = {2014},
date = {2014-01-01},
booktitle = {European Conf. on Artificial Intelligence. Workshop on Cognitive Robotics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2013
J. Gonzalez-Jimenez, C. Galindo, F. Melendez-Fernandez, J.R. Ruiz-Sarmiento
Building and Exploiting Maps in a Telepresence Robotic Application
10th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Reykjavic, Iceland, 2013.
@inproceedings{gonzalezjimenez2013b,
title = {Building and Exploiting Maps in a Telepresence Robotic Application},
author = {Gonzalez-Jimenez, Javier and Galindo, Cipriano and Melendez-Fernandez, Francisco and Ruiz-Sarmiento, J. R.},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/cgalindo/papers/draft_ICINCO2013-Map.pdf},
year = {2013},
date = {2013-01-01},
booktitle = {10th International Conference on Informatics in Control, Automation and Robotics (ICINCO)},
address = {Reykjavic, Iceland},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J. Gonzalez-Jimenez, J.R. Ruiz-Sarmiento, C. Galindo
Improving 2D Reactive Navigators with Kinect
10th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Reykjavic, Iceland, 2013.
@inproceedings{gonzalezjimenez2013c,
title = {Improving 2D Reactive Navigators with Kinect},
author = {Gonzalez-Jimenez, Javier and Ruiz-Sarmiento, J. R. and Galindo, Cipriano},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/cgalindo/papers/draft_ICINCO2013-Kinect.pdf},
year = {2013},
date = {2013-01-01},
booktitle = {10th International Conference on Informatics in Control, Automation and Robotics (ICINCO)},
address = {Reykjavic, Iceland},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2012
J. Gonzalez-Jimenez, C. Galindo, J.R. Ruiz-Sarmiento
Technical Improvements of the Giraff Telepresence Robot Based on Users’ Evaluation
2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012.
@inproceedings{gonzalezjimenez2012,
title = {Technical Improvements of the Giraff Telepresence Robot Based on Users’ Evaluation},
author = {Gonzalez-Jimenez, Javier and Galindo, Cipriano and Ruiz-Sarmiento, J. R.},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/cgalindo/papers/draft_RoMan2012.pdf},
doi = {10.1109/ROMAN.2012.6343854},
year = {2012},
date = {2012-09-01},
booktitle = {2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
National conferences (13)
2026
J. Serrano-Mesa, J. Moncada-Ramirez, P. Ojeda, G. Ambrosio, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Mapeo semántico de vocabulario abierto mediante 3D Gaussian Splatting
Jornadas de Automática CEA, Córdoba, 2026.
@inproceedings{SerranoMesa2026,
title = {Mapeo semántico de vocabulario abierto mediante 3D Gaussian Splatting},
author = {Serrano-Mesa, Jorge and Moncada-Ramirez, Jesús and Ojeda, Pepe and Ambrosio, Gregorio and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://revistas.udc.es/index.php/JA_CEA/article/view/13803/9700},
doi = {10.17979/ja-cea.2026.47.13803},
year = {2026},
date = {2026-09-01},
urldate = {2026-09-01},
booktitle = {Jornadas de Automática CEA, Córdoba},
journal = {XLVII Jornadas de Automática, Córdoba, Visión por Computador},
number = {47},
abstract = {<jats:p>Open-vocabulary semantic maps are key to the autonomy of mobile robots in complex, unstructured environments. Compared to traditional techniques, open-vocabulary semantic maps represent a significant improvement, as they are not limited by a predefined set of categories. For their construction, geometric representations based on 3D Gaussian Splatting enriched with low-level semantic descriptors have gained popularity. However, robotic operations demand high-level labels, such as textual descriptions and categories. This work presents an evolution of OpenSplat3D that integrates Large Vision-Language Models (LVLMs) for the description and categorization of object instances, whose consistency is validated within the latent space of Vision-Language Models (VLMs). Additionally, the geometric reconstruction has been optimized through dense initialization and the use of a depth-based loss function. Both proposals are validated using the Replica and ScanNet datasets, yielding accurate, semantically enriched maps.</jats:p>},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
<jats:p>Open-vocabulary semantic maps are key to the autonomy of mobile robots in complex, unstructured environments. Compared to traditional techniques, open-vocabulary semantic maps represent a significant improvement, as they are not limited by a predefined set of categories. For their construction, geometric representations based on 3D Gaussian Splatting enriched with low-level semantic descriptors have gained popularity. However, robotic operations demand high-level labels, such as textual descriptions and categories. This work presents an evolution of OpenSplat3D that integrates Large Vision-Language Models (LVLMs) for the description and categorization of object instances, whose consistency is validated within the latent space of Vision-Language Models (VLMs). Additionally, the geometric reconstruction has been optimized through dense initialization and the use of a depth-based loss function. Both proposals are validated using the Replica and ScanNet datasets, yielding accurate, semantically enriched maps.</jats:p>
2025
A. Torremocha-Doblas, P. Ojeda, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Application of Large Vision-Language Models for Semantic Map Refinement in Mobile Robotics
Simposio CEA de Robótica, Bioingeniería, Visión Artificial y Automática Marina, Almería (Spain), 2025.
@inproceedings{torremocha2025rbvm,
title = {Application of Large Vision-Language Models for Semantic Map Refinement in Mobile Robotics},
author = {Torremocha-Doblas, Adrian and Ojeda, Pepe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.64117/simposioscea.v1i1.56},
year = {2025},
date = {2025-06-01},
booktitle = {Simposio CEA de Robótica, Bioingeniería, Visión Artificial y Automática Marina},
volume = {1},
number = {1},
address = {Almería (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
M. Decena-Giménez, J. Moncada-Ramirez, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Semantic Segmentation of Object Instances Using an Open Vocabulary
Simposio CEA de Robótica, Bioingeniería, Visión Artificial y Automática Marina, Almería (Spain), 2025.
@inproceedings{decena2025rbvm,
title = {Semantic Segmentation of Object Instances Using an Open Vocabulary},
author = {Decena-Giménez, Macorís and Moncada-Ramirez, Jesús and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.64117/simposioscea.v1i1.38},
year = {2025},
date = {2025-06-01},
urldate = {2025-06-01},
booktitle = {Simposio CEA de Robótica, Bioingeniería, Visión Artificial y Automática Marina},
volume = {1},
number = {1},
address = {Almería (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2024
J. Moncada-Ramirez, J.R. Ruiz-Sarmiento, J.L. Matez-Bandera, J. Gonzalez-Jimenez
Large Models for Semantic Mapping in Mobile Robotics
XLV Jornadas de Automática, Málaga (Spain), 2024.
@inproceedings{moncada2024ja,
title = {Large Models for Semantic Mapping in Mobile Robotics},
author = {Moncada-Ramirez, Jesús and Ruiz-Sarmiento, J. R. and Matez-Bandera, J. L and Gonzalez-Jimenez, Javier},
url = {https://revistas.udc.es/index.php/JA_CEA/article/view/10940/8379},
doi = {10.17979/ja-cea.2024.45.10940},
year = {2024},
date = {2024-09-01},
booktitle = {XLV Jornadas de Automática},
address = {Málaga (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
G. Ambrosio, J.L. Matez-Bandera, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Linux Container-based Environment for the Development of Robotic Applications
XLV Jornadas de Automática, Málaga (Spain), 2024.
@inproceedings{goyo2024ja,
title = {Linux Container-based Environment for the Development of Robotic Applications},
author = {Ambrosio, Gregorio and Matez-Bandera, J. L and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://revistas.udc.es/index.php/JA_CEA/article/view/10943/8381},
doi = {10.17979/ja-cea.2024.45.10943},
year = {2024},
date = {2024-09-01},
booktitle = {XLV Jornadas de Automática},
address = {Málaga (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
A. Cañete, E. Quemada-Torres, J.R. Ruiz-Sarmiento, F.A. Moreno, J. Gonzalez-Jimenez
Multimodal System for the Orientation of Mobile Robots Towards its Interlocutor
XLV Jornadas de Automática, Málaga (Spain), 2024.
@inproceedings{ant2024ja,
title = {Multimodal System for the Orientation of Mobile Robots Towards its Interlocutor},
author = {Cañete, A. and Quemada-Torres, E. and Ruiz-Sarmiento, J. R. and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
url = {https://revistas.udc.es/index.php/JA_CEA/article/view/10939/8378},
doi = {10.17979/ja-cea.2024.45.10939},
year = {2024},
date = {2024-09-01},
urldate = {2024-09-01},
booktitle = {XLV Jornadas de Automática},
address = {Málaga (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2022
A. Galeote-Luque, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Método de Odometría Basada en Planos para Cámaras de Profundidad
XII Jornadas Nacionales de Robótica, Educación en Automática, y Bioingeniería, Malaga, Spain, 2022, ISBN: ISBN 978-84-09-41095-8.
@inproceedings{galeote2022jreb,
title = {Método de Odometría Basada en Planos para Cámaras de Profundidad},
author = {Galeote-Luque, Andres and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/24228},
isbn = {ISBN 978-84-09-41095-8},
year = {2022},
date = {2022-05-01},
booktitle = {XII Jornadas Nacionales de Robótica, Educación en Automática, y Bioingeniería},
address = {Malaga, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2019
S.F. Baltanas-Molero, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Colección de Jupyter Notebooks para cursos de robótica móvil
XL Jornadas de Automática, pp. 663-670, Ferrol, Spain, 2019.
@inproceedings{baltanas2019ja,
title = {Colección de Jupyter Notebooks para cursos de robótica móvil},
author = {Baltanas-Molero, Samuel-Felipe and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.17979/spudc.9788497497169.663},
year = {2019},
date = {2019-09-01},
booktitle = {XL Jornadas de Automática},
pages = {663-670},
address = {Ferrol, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2018
J.R. Ruiz-Sarmiento, J. Monroy, F.A. Moreno, J. Gonzalez-Jimenez
Tutorial para el Reconocimiento de Objetos Basado en Características Empleando Herramientas Python
XXXIX Jornadas de Automática, Badajoz (Spain), 2018, ISBN: 978-84-09-04460-3.
@inproceedings{raulruiz_JA_2018(inspanish),
title = {Tutorial para el Reconocimiento de Objetos Basado en Características Empleando Herramientas Python},
author = {Ruiz-Sarmiento, J. R. and Monroy, Javier and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_raul_JA_tutorial_reconocimiento_objetos.pdf},
isbn = {978-84-09-04460-3},
year = {2018},
date = {2018-01-01},
booktitle = {XXXIX Jornadas de Automática},
address = {Badajoz (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2017
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Modelado del Contexto Geométrico para el Reconocimiento de Objetos
Actas de las XXXVIII Jornadas de Automática, Gijón, Spain, 2017.
@inproceedings{raul2017ja,
title = {Modelado del Contexto Geométrico para el Reconocimiento de Objetos},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017-raul-JA-Modelado_del_Contexto_Geometrico.pdf},
year = {2017},
date = {2017-09-01},
booktitle = {Actas de las XXXVIII Jornadas de Automática},
address = {Gijón, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2016
D. Zuñiga-Noël, J.R. Ruiz-Sarmiento, J. Gonzalez-Jimenez
Detección de Lugares con Camaras RGB-D. Aplicación a Cierre de Bucles en SLAM
Actas de las XXXVII Jornadas de Automática, Madrid, 2016.
@inproceedings{david2016ja,
title = {Detección de Lugares con Camaras RGB-D. Aplicación a Cierre de Bucles en SLAM},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016-david-Jornadas_CEA-Deteccion_de_Lugares_con_Camaras_RGB-D.pdf},
year = {2016},
date = {2016-09-01},
booktitle = {Actas de las XXXVII Jornadas de Automática},
address = {Madrid},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2011
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez, J.L. Blanco
Navegación Reactiva de un Robot Móvil usando Kinect
Actas ROBOT 2011, 2011.
@inproceedings{ruizsarmiento2011nrr,
title = {Navegación Reactiva de un Robot Móvil usando Kinect},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier and Blanco, José-Luis},
url = {http://ingmec.ual.es/~jlblanco/papers/robot2011ruiz-sarmiento.pdf},
year = {2011},
date = {2011-11-01},
booktitle = {Actas ROBOT 2011},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Cámaras basadas en tiempo de vuelo. Uso en la mejora de métodos de detección de caras.
XXXII Jornadas de Automática, Sevilla., 2011.
@inproceedings{ruizsarmiento2011c,
title = {Cámaras basadas en tiempo de vuelo. Uso en la mejora de métodos de detección de caras.},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2011/2011-raul-Jornadas_CEA-Camaras_basadas_en_Tiempo_de_Vuelo.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {XXXII Jornadas de Automática, Sevilla.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
PhD Theses (2)
2024
A. Galeote-Luque, J. Gonzalez-Jimenez, J.R. Ruiz-Sarmiento
Advances in 3D Range Sensors Odometry for Mobile Robotics
University of Málaga, 2024.
@phdthesis{galeote2024tesis,
title = {Advances in 3D Range Sensors Odometry for Mobile Robotics},
author = {Galeote-Luque, Andres and Gonzalez-Jimenez, Javier and Ruiz-Sarmiento, J. R.},
url = {https://hdl.handle.net/10630/31623},
year = {2024},
date = {2024-01-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2016
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Probabilistic Techniques in Semantic Mapping for Mobile Robotics
University of Málaga, 2016.
@phdthesis{ruizsarmiento2016b,
title = {Probabilistic Techniques in Semantic Mapping for Mobile Robotics},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016-raul-probabilistic_techniques_in_semantic_mapping_for_mobile_robotics.pdf
http://hdl.handle.net/10630/14320},
year = {2016},
date = {2016-11-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Technical Reports (1)
2013
J.R. Ruiz-Sarmiento, C. Galindo, J. Gonzalez-Jimenez
Experimental Study of the Performance of the Kinect Range Camera for Mobile Robotics
Universidad de Malaga, Andalucia Tech, Departamento de Ingenieria de Sistemas y Automatica 2013.
@techreport{ruizsarmiento2013,
title = {Experimental Study of the Performance of the Kinect Range Camera for Mobile Robotics},
author = {Ruiz-Sarmiento, J. R. and Galindo, Cipriano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/cgalindo/papers/tech-kinect.pdf},
year = {2013},
date = {2013-01-01},
institution = {Universidad de Malaga, Andalucia Tech, Departamento de Ingenieria de Sistemas y Automatica},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
