Rubén Gómez Ojeda PhD. student in Computer Vision and Mobile Robotics
Welcome!!
I am Rubén Gómez Ojeda, a PhD student in the Machine Perception and Intelligent Robotics group (MAPIR) at the University of Málaga (Spain). My research interests include:
Computer Vision: Visual odometry and SLAM, and Deep Learning based Place Recognition
Mobile Robotics: Obstacle avoidance for autonomous vehicles.
Unmanned Aerial Vehicles (UAVs).
Sensor Fusion.
Contents
Brief CV
I was born in Málaga (Spain) in 1988. I received a B.Sc-M.Sc in “Ingeniería Industrial” (a general engineering which covers mechanics, electricity, electronic, computer science, etc.) from the University of Málaga in 2012. In October 2013 I joined the Machine Perception and Intelligent Robotics (MAPIR) group as a Researcher in Computer Vision. I received the M.Sc in Mechatronics in 2014. I received a grant (DPI2014-55826-R) from the National (Spanish) Plan of Research to do a 4-year PhD under the supervision of Prof. Javier González-Jiménez, which I started in November 2015. From October 2016 to February 2017 I was a visting researcher at the Robotics and Perception Group of the University of Zurich, under the supervision of Prof. Davide Scaramuzza. From June to September 2018 I was a research intern at Oculus in Zurich.
Research
Geometric-based Line Segment Tracking for HDR Stereo Sequences
In this work, we propose a purely geometrical approach for the robust matching of line segments for challenging stereo streams with severe illumination changes or High Dynamic Range (HDR) environments. To that purpose, we exploit the univocal nature of the matching problem, i.e. every observation must be corresponded with a single feature or not corresponded at all. We state the problem as a sparse, convex, L1-minimization of the matching vector regularized by the geometric constraints. This formulation allows for the robust tracking of line segments along sequences where traditional appearance-based matching techniques tend to fail due to dynamic changes in illumination conditions. Moreover, the proposed matching algorithm also results in a considerable speed-up of previous state of the art techniques making it suitable for real-time applications such as Visual Odometry (VO). This, of course, comes at expense of a slightly lower number of matches in comparison with appearance based methods, and also limits its application to continuous video sequences, as it is rather constrained to small pose increments between consecutive frames.We validate the claimed advantages by first evaluating the matching performance in challenging video sequences, and then testing the method in a benchmarked point and line based VO algorithm.
In One of the main open challenges in visual odometry (VO) is the robustness to difficult illumination conditions or high dynamic range (HDR) environments. We address this problem from a deep learning perspective, for which we propose two different deep networks: a very deep model consisting of both CNNs and LSTM, and another one of small size capable of executing in real-time on a GPU. Both networks transform a sequence of RGB images into more informative ones, while also being robust to changes in illumination, exposure time, gamma correction, etc. We validate the enhanced representations by evaluating the sequences produced by the two architectures in several state-of-art VO algorithms, such as ORB-SLAM and DSO.
In this work we propose a combined approach to stereo visual SLAM based on the simultaneous employment of both point and line segment features, as in our previous approaches to Visual Odometry, that is capable of working robustly in a wide variety of scenarios. As a consequence, we also obtain meaningful maps that can be further employed to extract valuable information from structured scenarios. The camera motion is recovered through non-linear minimization of the projection errors of both point and line segment features, with an ad-hoc implementation of the combined algorithm to solve the bundle adjustment problem for this particular case.
Place recognition is still an open problem in computer vision, and its difficulty increases under changes in the scenario, viewpoint, illumination or weather condition. We propose a Convolutional Neural Network (CNN) with the purpose of recognize the same location under severe weather or illumination variations, seasonal changes, etc. In contrast to previous approaches which rely on visual descriptors, our algorithm works with the complete image, reducing unnecessary errors induced by posterior feature matching processes by providing a better estimate of place similarity.
Monocular Visual Odometry
In this work, we extend a popular semi-direct approach to monocular visual odometry known as SVO to work with line segments, hence obtaining a more robust system capable of dealing with both textured and structured environments. The proposed odometry system allows for the fast tracking of line segments since it eliminates the necessity of continuously extracting and matching features between subsequent frames. The method, of course, has a higher computational burden than the original SVO, but it still runs with frequencies of 60Hz on a personal computer while performing robustly in a wider variety of scenarios.
A common strategy to stereo visual odometry (SVO), known as feature-based, tracks some relevant features (traditionally keypoints) in a sequence of stereo images, and then estimates the pose increment between those frames by imposing some rigid-body constraints between the features. However, in low-textured scenes it is often difficult to encounter a large set of point features, or it may happen that they are not well distributed over the image, so that the behavior of these point-based algorithms deteriorate. For that, we propose a probabilistic SVO algorithm based on the combination of both keypoints and line segments, since they provide complementary information, and hence capable of working in a wide variety of environments.
@article{zuniga2020ijrr,
title = {The UMA-VI dataset: Visual–inertial odometry in low-textured and dynamic illumination environments},
author = {Zuñiga-Noël, David and Jaenal, Alberto and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2020/2020_The_UMA-VI_dataset_Visual%e2%80%93inertial_odometry_in_low-textured_and_dynamic_illumination_environments.pdf
https://journals.sagepub.com/doi/10.1177/0278364920938439},
doi = {10.1177/0278364920938439},
year = {2020},
date = {2020-07-01},
journal = {The International Journal of Robotics Research},
abstract = {This article presents a visual–inertial dataset gathered in indoor and outdoor scenarios with a handheld custom sensor rig, for over 80 min in total. The dataset contains hardware-synchronized data from a commercial stereo camera (Bumblebee®2), a custom stereo rig, and an inertial measurement unit. The most distinctive feature of this dataset is the strong presence of low-textured environments and scenes with dynamic illumination, which are recurrent corner cases of visual odometry and simultaneous localization and mapping (SLAM) methods. The dataset comprises 32 sequences and is provided with ground-truth poses at the beginning and the end of each of the sequences, thus allowing the accumulated drift to be measured in each case. We provide a trial evaluation of five existing state-of-the-art visual and visual–inertial methods on a subset of the dataset. We also make available open-source tools for evaluation purposes, as well as the intrinsic and extrinsic calibration parameters of all sensors in the rig. The dataset is available for download at https://mapir.isa.uma.es/work/uma-visual-inertial-dataset},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This article presents a visual–inertial dataset gathered in indoor and outdoor scenarios with a handheld custom sensor rig, for over 80 min in total. The dataset contains hardware-synchronized data from a commercial stereo camera (Bumblebee®2), a custom stereo rig, and an inertial measurement unit. The most distinctive feature of this dataset is the strong presence of low-textured environments and scenes with dynamic illumination, which are recurrent corner cases of visual odometry and simultaneous localization and mapping (SLAM) methods. The dataset comprises 32 sequences and is provided with ground-truth poses at the beginning and the end of each of the sequences, thus allowing the accumulated drift to be measured in each case. We provide a trial evaluation of five existing state-of-the-art visual and visual–inertial methods on a subset of the dataset. We also make available open-source tools for evaluation purposes, as well as the intrinsic and extrinsic calibration parameters of all sensors in the rig. The dataset is available for download at https://mapir.isa.uma.es/work/uma-visual-inertial-dataset
@article{zuniga2019ral,
title = {Automatic Multi-Sensor Extrinsic Calibration For Mobile Robots},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
doi = {10.1109/LRA.2019.2922618},
year = {2019},
date = {2019-07-01},
journal = {IEEE Robotics and Automation Letters},
volume = {4},
number = {3},
pages = {2862-2869},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{gomez2019pss,
title = {PL-SLAM: a Stereo SLAM System Through the Combination of Points and Line Segments},
author = {Gomez-Ojeda, Ruben and Moreno, Francisco-Angel and Zuñiga-Noël, David and Scaramuzza, Davide and Gonzalez-Jimenez, Javier},
url = {https://ieeexplore.ieee.org/abstract/document/8680013},
doi = {10.1109/TRO.2019.2899783},
issn = {1552-3098},
year = {2019},
date = {2019-06-01},
journal = {IEEE Transactions on Robotics},
volume = {35},
number = {3},
pages = {734 - 746},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{antequera2017,
title = {Appearance-Invariant Place Recognition by Discriminatively Training a Convolutional Neural Network},
author = {Lopez-Antequera, Manuel and Gomez-Ojeda, Ruben and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {http://www.sciencedirect.com/science/article/pii/S0167865517301381},
doi = {10.1016/j.patrec.2017.04.017},
issn = {0167-8655},
year = {2017},
date = {2017-06-01},
journal = {Pattern Recognition Letters},
volume = {92},
pages = {89-95},
abstract = {Abstract Visual place recognition is the task of automatically recognizing a previously visited location through its appearance, and plays a key role in mobile robotics and autonomous driving applications. The difficulty of recognizing a revisited location increases with appearance variations caused by weather, illumination or point of view changes. In this paper we present a convolutional neural network (CNN) embedding to perform place recognition, even under severe appearance changes. The network maps images to a low dimensional space where images from nearby locations map to points close to each other, despite differences in visual appearance caused by the aforementioned phenomena. In order for the network to learn the desired invariances, we train it with triplets of images selected from datasets which present a challenging variability in visual appearance. Our proposal is validated through extensive experimentation that reveals better performance than state-of-the-art methods. Importantly, though the training phase is computationally demanding, its online application is very efficient.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Visual place recognition is the task of automatically recognizing a previously visited location through its appearance, and plays a key role in mobile robotics and autonomous driving applications. The difficulty of recognizing a revisited location increases with appearance variations caused by weather, illumination or point of view changes. In this paper we present a convolutional neural network (CNN) embedding to perform place recognition, even under severe appearance changes. The network maps images to a low dimensional space where images from nearby locations map to points close to each other, despite differences in visual appearance caused by the aforementioned phenomena. In order for the network to learn the desired invariances, we train it with triplets of images selected from datasets which present a challenging variability in visual appearance. Our proposal is validated through extensive experimentation that reveals better performance than state-of-the-art methods. Importantly, though the training phase is computationally demanding, its online application is very efficient.
@unpublished{gomezojeda2017b,
title = {PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments},
author = {Gomez-Ojeda, Ruben and Zuñiga-Noël, David and Moreno, Francisco-Angel and Scaramuzza, Davide and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/rgomez/publications/tro17plslam.pdf
https://arxiv.org/abs/1705.09479},
year = {2017},
date = {2017-01-01},
note = {arXiv:1705.09479},
keywords = {},
pubstate = {published},
tppubtype = {unpublished}
}
@unpublished{Gomez-Ojeda2015,
title = {Training a Convolutional Neural Network for Appearance-Invariant Place Recognition},
author = {Gomez-Ojeda, Ruben and Lopez-Antequera, Manuel and Petkov, Nicolai and Gonzalez-Jimenez, Javier},
url = {http://arxiv.org/pdf/1505.07428v1.pdf
http://arxiv.org/abs/1505.07428},
year = {2015},
date = {2015-01-01},
abstract = {Place recognition is one of the most challenging problems in computer vision, and has become a key part in mobile robotics and autonomous driving applications for performing loop closure in visual SLAM systems. Moreover, the difficulty of recognizing a revisited location increases with appearance changes caused, for instance, by weather or illumination variations, which hinders the long-term application of such algorithms in real environments. In this paper we present a convolutional neural network (CNN), trained for the first time with the purpose of recognizing revisited locations under severe appearance changes, which maps images to a low dimensional space where Euclidean distances represent place dissimilarity. In order for the network to learn the desired invariances, we train it with triplets of images selected from datasets which present a challenging variability in visual appearance. The triplets are selected in such way that two samples are from the same location and the third one is taken from a different place. We validate our system through extensive experimentation, where we demonstrate better performance than state-of-art algorithms in a number of popular datasets.},
note = {http://arxiv.org/abs/1505.07428},
keywords = {},
pubstate = {published},
tppubtype = {unpublished}
}
Place recognition is one of the most challenging problems in computer vision, and has become a key part in mobile robotics and autonomous driving applications for performing loop closure in visual SLAM systems. Moreover, the difficulty of recognizing a revisited location increases with appearance changes caused, for instance, by weather or illumination variations, which hinders the long-term application of such algorithms in real environments. In this paper we present a convolutional neural network (CNN), trained for the first time with the purpose of recognizing revisited locations under severe appearance changes, which maps images to a low dimensional space where Euclidean distances represent place dissimilarity. In order for the network to learn the desired invariances, we train it with triplets of images selected from datasets which present a challenging variability in visual appearance. The triplets are selected in such way that two samples are from the same location and the third one is taken from a different place. We validate our system through extensive experimentation, where we demonstrate better performance than state-of-art algorithms in a number of popular datasets.
@inproceedings{jaenal2021improving,
title = {Improving Visual SLAM in Car-Navigated Urban Environments with Appearance Maps},
author = {Jaenal, Alberto and Zuñiga-Noël, David and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
doi = {10.1109/IROS45743.2020.9341451},
year = {2020},
date = {2020-01-01},
booktitle = {2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages = {4679-4685},
abstract = {This paper describes a method that corrects errors of a VSLAM-estimated trajectory for cars driving in GPS-denied environments, by applying constraints from public databases of geo-tagged images (Google Street View, Mapillary, etc). The method, dubbed Appearance-based Geo-Alignment for Simultaneous Localisation and Mapping (AGA-SLAM), encodes the available image database as an appearance map, which represents the space with a compact holistic descriptor for each image plus its associated geo-tag. The VSLAM trajectory is corrected on-line by incorporating constraints from the recognized places along the trajectory into a position-based optimization framework. The paper presents a seamless formulation to combine local and absolute metric observations with associations from Visual Place Recognition. The robustness of the holistic image descriptor to changes due to weather or illumination variations ensures a long-term consistent method to improve car localization. The proposed method has been extensively evaluated on more than 70 sequences from 4 different datasets, proving out its effectiveness and endurance to appearance challenges.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper describes a method that corrects errors of a VSLAM-estimated trajectory for cars driving in GPS-denied environments, by applying constraints from public databases of geo-tagged images (Google Street View, Mapillary, etc). The method, dubbed Appearance-based Geo-Alignment for Simultaneous Localisation and Mapping (AGA-SLAM), encodes the available image database as an appearance map, which represents the space with a compact holistic descriptor for each image plus its associated geo-tag. The VSLAM trajectory is corrected on-line by incorporating constraints from the recognized places along the trajectory into a position-based optimization framework. The paper presents a seamless formulation to combine local and absolute metric observations with associations from Visual Place Recognition. The robustness of the holistic image descriptor to changes due to weather or illumination variations ensures a long-term consistent method to improve car localization. The proposed method has been extensively evaluated on more than 70 sequences from 4 different datasets, proving out its effectiveness and endurance to appearance challenges.
@inproceedings{gomez2015calib,
title = {Extrinsic Calibration of a 2D Laser-Rangefinder and a Camera based on Scene Corners},
author = {Gomez-Ojeda, Ruben and Briales, Jesus and Fernández-Moral, Eduardo and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/rgomez/publications/icra15extrinsic.pdf},
doi = {10.1109/ICRA.2015.7139700},
year = {2015},
date = {2015-01-01},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
pages = {3611--3616},
address = {Seattle, USA},
organization = {IEEE},
abstract = {Robots are often equipped with 2D laser-
rangefinders (LRFs) and cameras since they complement well
to each other. In order to correctly combine measurements from
both sensors, it is required to know their relative pose, that is, to
solve their extrinsic calibration. In this paper we present a new
approach to such problem which relies on the observations of
an orthogonal trihedron which is profusely found as corners in
human-made scenarios. Thus, the method does not require any
specific pattern, which turns the calibration process fast and
simpler to perform. The estimated relative pose has proven
to be also very precise since it uses two different types of
constraints, line-to-plane and point-to-plane, as a result of a
richer configuration than previous proposals that relies on plane
or V-shaped patterns. Our approach is validated with synthetic
and real experiments, showing better performance than the
state-of-art methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Robots are often equipped with 2D laser-
rangefinders (LRFs) and cameras since they complement well
to each other. In order to correctly combine measurements from
both sensors, it is required to know their relative pose, that is, to
solve their extrinsic calibration. In this paper we present a new
approach to such problem which relies on the observations of
an orthogonal trihedron which is profusely found as corners in
human-made scenarios. Thus, the method does not require any
specific pattern, which turns the calibration process fast and
simpler to perform. The estimated relative pose has proven
to be also very precise since it uses two different types of
constraints, line-to-plane and point-to-plane, as a result of a
richer configuration than previous proposals that relies on plane
or V-shaped patterns. Our approach is validated with synthetic
and real experiments, showing better performance than the
state-of-art methods.
@inproceedings{zuniga_jjaa_17,
title = {Calibración Extrínseca de un Conjunto de Cáamaras RGB-D sobre un Robot Móvil},
author = {Zuñiga-Noël, David and Gomez-Ojeda, Ruben and Moreno, Francisco-Angel and Gonzalez-Jimenez, Javier},
year = {2017},
date = {2017-09-01},
booktitle = {XXXVIII Jornadas de Automática},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@phdthesis{gomezojeda2020,
title = {Robust Visual SLAM in Challenging Environments with Low-texture and Dynamic Illumination},
author = {Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/19479},
year = {2020},
date = {2020-01-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
