Visual odometry (VO), also known as egomotion, is the process of estimating the trajectory of a camera within a rigid environment by analyzing a sequence of images.
In general, VO is employed in both mobile robotics and autonomous driving applications since it overcomes the drawbacks presented by traditional solutions as wheel odometry, IMUs or GPS-based systems. Other applications include augmented reality and virtual reality, which often employ VO algorithms to localize the virtual devices. Here, we address the visual odometry problem from several perspectives by employing different sensors:
Contents
Monocular Cameras
PL-SVO: In this work, we extend a popular semi-direct approach to monocular visual odometry known as SVO to work with line segments, hence obtaining a more robust system capable of dealing with both textured and structured environments. The proposed odometry system allows for the fast tracking of line segments since it eliminates the necessity of continuously extracting and matching features between subsequent frames. The method, of course, has a higher computational burden than the original SVO, but it still runs with frequencies of 60Hz on a personal computer while performing robustly in a wider variety of scenarios.
Stereo Cameras
Stereo Visual Odometry with Points and Lines: A common strategy to stereo visual odometry (SVO), known as feature-based, tracks some relevant features (traditionally keypoints) in a sequence of stereo images, and then estimates the pose increment between those frames by imposing some rigid-body constraints between the features. However, in low-textured scenes it is often difficult to encounter a large set of point features, or it may happen that they are not well distributed over the image, so that the behavior of these point-based algorithms deteriorate. For that, we propose a probabilistic SVO algorithm based on the combination of both keypoints and line segments, since they provide complementary information, and hence capable of working in a wide variety of environments.
We have developed a new dense method to compute the odometry of a free-flying range sensor in real-time. The method applies the range flow constraint equation to sensed points in the temporal flow to derive the linear and angular velocity of the sensor in a rigid environment. A distinct feature of our method is that the same input data (range measurements) are exploited both to obtain the camera motion that minimizes the geometric error and to perform the warping within a coarse-to-fine scheme, which makes it applicable to any range sensor (not only to RGB-D cameras as other existing approaches). In particular, we have implemented and tested it for two different sensors: range cameras and laser scanners.
RF2O (Range Flow-based 2D Odometry for laser scanners): With a runtime of barely 1 millisecond, planar motion can be estimated with a negligible computational cost, which makes this method attractive for many robotic applications that are computationally demanding and might require real-time performance. Several experiments have been conducted to test the performance of our method, including qualitative and quantitative comparisons with wheel odometry, Point-to-Line ICP and the Polar Scan Matcher. Overall, our approach provides the most accurate results with the lowest runtime.
DIFODO (DIFferential ODOmetry for range cameras): It runs in real-time on a single CPU core. Comparisons have been performed with two state-of-the-art methods: Generalized-ICP and the RDVO. Experiments show that our approach clearly overperforms GICP which uses the same geometric input data, whereas it achieves similar results than RDVO, which requires both geometric and photometric data to work. Furthermore, experiments have been carried out to demonstrate that our approach is able to estimate fast motions precisely at 60 Hz running on a single CPU core (with QQVGA resolution).
Publications
Please refer to the following articles for further details:
@article{garciasalguero2022,
title = {A Tighter Relaxation for the Relative Pose Problem Between Cameras},
author = {Garcia-Salguero, Mercedes and Briales, Jesus and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2022/2022_mercedes_JMIV_tighter_rel_rpp_paper.pdf},
doi = {10.1007/s10851-022-01085-z},
year = {2022},
date = {2022-01-01},
journal = {Journal of Mathematical Imaging and Vision},
number = {64},
pages = {493-505},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{garciasalguero2021b,
title = {Fast and Robust Certifiable Estimation of the Relative Pose Between Two Calibrated Cameras},
author = {Garcia-Salguero, Mercedes and Gonzalez-Jimenez, Javier},
url = {https://arxiv.org/pdf/2101.08524.pdf},
doi = {10.1007/s10851-021-01044-0},
year = {2021},
date = {2021-06-01},
journal = {Journal of Mathematical Imaging and Vision},
volume = {63},
pages = {1036-1056},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{zuniga2020ijrr,
title = {The UMA-VI dataset: Visual–inertial odometry in low-textured and dynamic illumination environments},
author = {Zuñiga-Noël, David and Jaenal, Alberto and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2020/2020_The_UMA-VI_dataset_Visual%e2%80%93inertial_odometry_in_low-textured_and_dynamic_illumination_environments.pdf
https://journals.sagepub.com/doi/10.1177/0278364920938439},
doi = {10.1177/0278364920938439},
year = {2020},
date = {2020-07-01},
journal = {The International Journal of Robotics Research},
abstract = {This article presents a visual–inertial dataset gathered in indoor and outdoor scenarios with a handheld custom sensor rig, for over 80 min in total. The dataset contains hardware-synchronized data from a commercial stereo camera (Bumblebee®2), a custom stereo rig, and an inertial measurement unit. The most distinctive feature of this dataset is the strong presence of low-textured environments and scenes with dynamic illumination, which are recurrent corner cases of visual odometry and simultaneous localization and mapping (SLAM) methods. The dataset comprises 32 sequences and is provided with ground-truth poses at the beginning and the end of each of the sequences, thus allowing the accumulated drift to be measured in each case. We provide a trial evaluation of five existing state-of-the-art visual and visual–inertial methods on a subset of the dataset. We also make available open-source tools for evaluation purposes, as well as the intrinsic and extrinsic calibration parameters of all sensors in the rig. The dataset is available for download at https://mapir.isa.uma.es/work/uma-visual-inertial-dataset},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This article presents a visual–inertial dataset gathered in indoor and outdoor scenarios with a handheld custom sensor rig, for over 80 min in total. The dataset contains hardware-synchronized data from a commercial stereo camera (Bumblebee®2), a custom stereo rig, and an inertial measurement unit. The most distinctive feature of this dataset is the strong presence of low-textured environments and scenes with dynamic illumination, which are recurrent corner cases of visual odometry and simultaneous localization and mapping (SLAM) methods. The dataset comprises 32 sequences and is provided with ground-truth poses at the beginning and the end of each of the sequences, thus allowing the accumulated drift to be measured in each case. We provide a trial evaluation of five existing state-of-the-art visual and visual–inertial methods on a subset of the dataset. We also make available open-source tools for evaluation purposes, as well as the intrinsic and extrinsic calibration parameters of all sensors in the rig. The dataset is available for download at https://mapir.isa.uma.es/work/uma-visual-inertial-dataset
@article{gomez2019pss,
title = {PL-SLAM: a Stereo SLAM System Through the Combination of Points and Line Segments},
author = {Gomez-Ojeda, Ruben and Moreno, Francisco-Angel and Zuñiga-Noël, David and Scaramuzza, Davide and Gonzalez-Jimenez, Javier},
url = {https://ieeexplore.ieee.org/abstract/document/8680013},
doi = {10.1109/TRO.2019.2899783},
issn = {1552-3098},
year = {2019},
date = {2019-06-01},
journal = {IEEE Transactions on Robotics},
volume = {35},
number = {3},
pages = {734 - 746},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{jaimez_tro_2018,
title = {Robust Planar Odometry based on Symmetric Range Flow and Multi-Scan Alignment},
author = {Jaimez, Mariano and Monroy, Javier and Lopez-Antequera, Manuel and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018-jaimez_TRO_Robust_Planar_Odometryy.pdf},
doi = {10.1109/TRO.2018.2861911},
issn = {1552-3098},
year = {2018},
date = {2018-01-01},
journal = {IEEE Transactions on Robotics},
pages = {1623--1635},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{moreno2016,
title = {A constant-time SLAM back-end in the continuum between global mapping and submapping: application to visual stereo SLAM},
author = {Moreno, Francisco-Angel and Blanco, José-Luis and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/famoreno/papers/pdfs/moreno2015ssrba_draft.pdf},
doi = {10.1177/0278364915619238},
issn = {0278-3649},
year = {2016},
date = {2016-08-01},
journal = {International Journal of Robotics Research},
volume = {35},
number = {9},
pages = {1036 -- 1056},
abstract = {This work addresses the development and application of a novel approach, called Sparser Relative Bundle Adjustment (SRBA), which exploits the inherent flexibility of the relative BA (RBA) framework to devise a continuum of strategies, ranging from RBA with linear graphs to classic BA in global coordinates, where submapping with local maps emerges as a natural intermediate solution. This method leads to graphs that can be optimized in bounded-time even at loop
closures, regardless of the loop length. Furthermore, it is shown that the pattern in which relative coordinate variables
are defined among keyframes has a significant impact on the graph optimization problem. By using the proposed
scheme, optimization can be done more efficiently than in standard RBA, allowing the optimization of larger local
maps for any given maximum computational cost. The main algorithms involved in the graph management, along
with their complexity analyses, are presented to prove their bounded-time nature. One key advance of the present
work is the demonstration that, under mild assumptions, the spanning trees for every single keyframe in the map
can be incrementally built by a constant-time algorithm, even for arbitrary graph topologies. We validate our proposal
within the scope of visual stereo SLAM by developing a complete system that includes a front-end that seamlessly
integrates several state-of-the-art computer vision techniques such as ORB features and bag-of-words, along with a
decision scheme for keyframe insertion and a SRBA-based back-end that operates as graph optimizer. Finally, a set of
experiments in both indoor and outdoor conditions is presented to test the capabilities of this approach. Open-source
implementations of the SRBA back-end and the stereo front-end have been released online.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This work addresses the development and application of a novel approach, called Sparser Relative Bundle Adjustment (SRBA), which exploits the inherent flexibility of the relative BA (RBA) framework to devise a continuum of strategies, ranging from RBA with linear graphs to classic BA in global coordinates, where submapping with local maps emerges as a natural intermediate solution. This method leads to graphs that can be optimized in bounded-time even at loop
closures, regardless of the loop length. Furthermore, it is shown that the pattern in which relative coordinate variables
are defined among keyframes has a significant impact on the graph optimization problem. By using the proposed
scheme, optimization can be done more efficiently than in standard RBA, allowing the optimization of larger local
maps for any given maximum computational cost. The main algorithms involved in the graph management, along
with their complexity analyses, are presented to prove their bounded-time nature. One key advance of the present
work is the demonstration that, under mild assumptions, the spanning trees for every single keyframe in the map
can be incrementally built by a constant-time algorithm, even for arbitrary graph topologies. We validate our proposal
within the scope of visual stereo SLAM by developing a complete system that includes a front-end that seamlessly
integrates several state-of-the-art computer vision techniques such as ORB features and bag-of-words, along with a
decision scheme for keyframe insertion and a SRBA-based back-end that operates as graph optimizer. Finally, a set of
experiments in both indoor and outdoor conditions is presented to test the capabilities of this approach. Open-source
implementations of the SRBA back-end and the stereo front-end have been released online.
@article{briales2015p3oa,
title = {A minimal closed-form solution for the Perspective Three orthogonal Angles (P3oA) problem. Application to visual odometry.},
author = {Briales, Jesus and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/jbriales/publications/JMIV16.pdf},
doi = {10.1007/s10851-015-0620-x},
year = {2016},
date = {2016-07-01},
journal = {Journal of Mathematical Imaging and Vision},
volume = {55},
number = {3},
pages = {266-283},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{jaimez2015c,
title = {Fast Visual Odometry for 3-D Range Sensors},
author = {Jaimez, Mariano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/mjaimez/Papers/Jaimez_and_Gonzalez_DIFODO_2015.pdf},
doi = {10.1109/TRO.2015.2428512},
year = {2015},
date = {2015-01-01},
journal = {IEEE Transactions on Robotics},
volume = {31},
number = {4},
pages = {809 - 822},
abstract = {This paper presents a new dense method to compute the odometry of a free-flying range sensor in real time. The method applies the range flow constraint equation to sensed points in the temporal flowto derive the linear and angular velocity of the sensor in a rigid environment. Although this approach is applicable to any range sensor, we particularize its formulation to estimate the 3-D motion of a range camera. The proposed algorithm is tested with different image resolutions and compared with two state-of-the-art methods: generalized iterative closest point (GICP) [1] and robust dense visual odometry (RDVO) [2]. Experiments show that our approach clearly overperforms GICP which uses the same geometric input data, whereas it achieves results similar to RDVO, which requires both geometric and photometric data towork. Furthermore, experiments are carried out to demonstrate that our approach is able to estimate fast motions at 60 Hz running on a single CPU core, a performance that has never been reported in the literature. The algorithm is available online under an open source license so that the robotic community can benefit from it.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents a new dense method to compute the odometry of a free-flying range sensor in real time. The method applies the range flow constraint equation to sensed points in the temporal flowto derive the linear and angular velocity of the sensor in a rigid environment. Although this approach is applicable to any range sensor, we particularize its formulation to estimate the 3-D motion of a range camera. The proposed algorithm is tested with different image resolutions and compared with two state-of-the-art methods: generalized iterative closest point (GICP) [1] and robust dense visual odometry (RDVO) [2]. Experiments show that our approach clearly overperforms GICP which uses the same geometric input data, whereas it achieves results similar to RDVO, which requires both geometric and photometric data towork. Furthermore, experiments are carried out to demonstrate that our approach is able to estimate fast motions at 60 Hz running on a single CPU core, a performance that has never been reported in the literature. The algorithm is available online under an open source license so that the robotic community can benefit from it.
@inproceedings{jaimez2017,
title = {Fast Odometry and Scene Flow from RGB-D Cameras based on Geometric Clustering},
author = {Jaimez, Mariano and Kerl, Christian and Gonzalez-Jimenez, Javier and Cremers, Daniel},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/mjaimez/Papers/Jaimez_et_al_VOSF_2017.pdf},
year = {2017},
date = {2017-01-01},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
address = {Singapore},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{Jaimez_et_al_ICRA2016,
title = {Planar Odometry from a Radial Laser Scanner. A Range Flow-based Approach},
author = {Jaimez, Mariano and Monroy, Javier and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016_Jaimez_ICRA_RF2O.pdf},
doi = {10.1109/ICRA.2016.7487647},
year = {2016},
date = {2016-06-01},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
pages = {4479-4485},
address = {Stockholm, Sweden},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{moreno2013erode,
title = {ERODE: An Efficient and Robust Outlier Detector and its Application to Stereovisual Odometry},
author = {Moreno, Francisco-Angel and Blanco, José-Luis and Gonzalez-Jimenez, Javier},
url = {http://ingmec.ual.es/~jlblanco/papers/icra2013_erode.pdf},
doi = {10.1109/ICRA.2013.6631245},
isbn = {978-1-4673-5641-1},
year = {2013},
date = {2013-05-01},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
pages = {4676 -- 4682},
publisher = {IEEE},
address = {Karlsruhe},
abstract = {This paper presents ERODE, an efficient outlier detector with a quality similar to that of standard RANSAC but at a fraction of its computational cost. In contrast to RANSAC-based methods which follow a hypothesis-and-verify approach, ERODE employs instead the whole set of observations together with a robust kernel to perform robustified least-squares minimization. Our proposal has important practical applications among computer vision problems, which we demonstrate with stereovisual odometry experiments with both simulated and real data.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents ERODE, an efficient outlier detector with a quality similar to that of standard RANSAC but at a fraction of its computational cost. In contrast to RANSAC-based methods which follow a hypothesis-and-verify approach, ERODE employs instead the whole set of observations together with a robust kernel to perform robustified least-squares minimization. Our proposal has important practical applications among computer vision problems, which we demonstrate with stereovisual odometry experiments with both simulated and real data.
@inproceedings{galeote2022jreb,
title = {Método de Odometría Basada en Planos para Cámaras de Profundidad},
author = {Galeote-Luque, Andres and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/24228},
isbn = {ISBN 978-84-09-41095-8},
year = {2022},
date = {2022-05-01},
booktitle = {XII Jornadas Nacionales de Robótica, Educación en Automática, y Bioingeniería},
address = {Malaga, Spain},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@phdthesis{galeote2024tesis,
title = {Advances in 3D Range Sensors Odometry for Mobile Robotics},
author = {Galeote-Luque, Andres and Gonzalez-Jimenez, Javier and Ruiz-Sarmiento, J. R.},
url = {https://hdl.handle.net/10630/31623},
year = {2024},
date = {2024-01-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
@phdthesis{briales2021,
title = {Global Optimality via Tight Convex Relaxations for Pose Estimation in Geometric 3D Computer Vision},
author = {Briales, Jesus and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/22918},
year = {2021},
date = {2021-01-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
@phdthesis{jaimez2017c,
title = {Motion Estimation, 3D Reconstruction and Navigation with Range Sensors},
author = {Jaimez, Mariano and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2017/2017_jaimez_PhDthesis_dual_nosummary.pdf
https://hdl.handle.net/10630/16257},
year = {2017},
date = {2017-11-01},
school = {Universidad de Málaga & Technische Universität München},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
@phdthesis{moreno2015,
title = {Stereo Visual SLAM for Mobile Robots Navigation},
author = {Moreno, Francisco-Angel and Blanco, José-Luis and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/famoreno/papers/thesis/FAMD_thesis.pdf
http://hdl.handle.net/10630/9859},
year = {2015},
date = {2015-04-01},
school = {University of Malaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
