Extrinsic sensor calibration is the estimation of the relative transformation between different sensors. This is a key step for sensor fusion.
Mobile robots tend to accomodate numerous sensors. These sensors can be of different nature (e.g. cameras, RGB-D devices, laser rangefinders, etc.) and, individually, serve a wide range of different purposes. The combination of the information resulting from separate sensors, also known as sensor fusion, boost the synergy of the sensory system and may improve input information or even enable new uses not possible with original data. Since each sensor has its own coordinate system wrt which received data is described, a key step for sensor fusion is to perform an accurate and trustworthy extrinsic calibration between sensors, preferrably accompanied by an analysis of its uncertainty.
Contents
Camera – LRF calibration based on scene corners
Cameras and 2D laser-rangefinders (LRFs) are devices commonly used on robots and complement well to each other: Cameras provide projective data only, whereas 2D LRFs provide range measurements. The combination of these two sensors give raise to potential applications, such as
pointcloud coloring
extraction of semantic data
improvement of visual odometry algorithms
The observation of an orthogonal trihedron, which can be profusely found as corners in human-made scenes, provides enough data to solve the calibration task without the need of any other specific calibration patern:
From the observation of the trihedron, both line-to-plane and point-to-plane constraints can be imposed on the calibration parameters. The nature of constraints allows us to decouple rotation computation from that of translation.
The treatment of extracted features and the computation of the necessary geometric data is suitable for automation.
All captured data can be input in a Maximum Likelihood Estimation (MLE) optimization process, properly weighted accordingly to the data certainty
Calibration based on the observation of scene corners
Extrinsic calibration of a set of 2D LRFs
The integration of several 2D laser rangefinders in a vehicle is a common resource employed for 3D mapping, obstacle detection and navigation. In this work we present a novel solution for the extrinsic calibration of a set of at least three laser scanners from the information provided by the sensor measurements. This method only requires the lasers to observe a common planar surface from different orientations, thus there is no need of any specific calibration pattern. This calibration technique can be used with almost any geometric sensor configuration (except for sensors scanning parallel planes), and constitutes a versatile solution that is accurate, fast and easy to apply.
Extrinsic calibration of a set of range cameras
The integration of several range cameras in a mobile platform is useful for applications in mobile robotics and autonomous vehicles that require a large field of view.The method that we present serves to calibrate two or more range cameras in an arbitrary configuration, requiring only to observe one plane from different viewpoints. The conditions to solve the problem are studied, and several practical examples are presented covering different geometric configurations, including an omnidirectional RGBD sensor composed of 8 range cameras. The quality of this calibration is evaluated with several experiments that demonstrate an improvement of accuracy over design parameters, while providing a versatile solution that is extremely fast and easy to apply.
Publications
Please refer to the following articles for further details:
@article{zuniga2019ral,
title = {Automatic Multi-Sensor Extrinsic Calibration For Mobile Robots},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gomez-Ojeda, Ruben and Gonzalez-Jimenez, Javier},
doi = {10.1109/LRA.2019.2922618},
year = {2019},
date = {2019-07-01},
journal = {IEEE Robotics and Automation Letters},
volume = {4},
number = {3},
pages = {2862-2869},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
@article{gonzalezjimenez2016,
title = {An Automated Surveying and Marking System for Continuous Setting-out of Tunnels},
author = {Gonzalez-Jimenez, Javier and Arevalo-Espejo, Vicente and Galindo, Cipriano and Ruiz-Sarmiento, J. R.},
url = {https://mapir.isa.uma.es/papersrepo/2016/2016-jgonzalez-CACIE-TCS.pdf},
doi = {10.1111/mice.12158},
year = {2016},
date = {2016-03-01},
journal = {Computer-Aided Civil and Infrastructure Engineering},
volume = {31},
number = {3},
pages = {219--228},
abstract = {Setting-out is a crucial and frequently repeated process in construction and civil engineering. It is carried out by qualified operators who, making use of surveying stations, identify reference points to guide workers in their tasks. In this paper, we focus on the particular case of setting-out road/train tunnels and propose a system that automatically performs the setting-out operation of the tunnel section to be perforated. The presented system, called Tunnel Continuous Setout (TCS), integrates a scanning device that surveys the excavation front and a laser projector that continuously displays the actual tunnel section computed as the intersection between the surveyed terrain and the planned tunnel section. Thus, the topographer intervention is only required for the precise positioning of the TCS device at the beginning of each working stretch, which depends on the operational range of its components (limited to 25 meters in the current implementation). A prototype of the TCS system has been employed in a real construction site proving its benefits and advantages with respect to the traditional setting-out techniques.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Setting-out is a crucial and frequently repeated process in construction and civil engineering. It is carried out by qualified operators who, making use of surveying stations, identify reference points to guide workers in their tasks. In this paper, we focus on the particular case of setting-out road/train tunnels and propose a system that automatically performs the setting-out operation of the tunnel section to be perforated. The presented system, called Tunnel Continuous Setout (TCS), integrates a scanning device that surveys the excavation front and a laser projector that continuously displays the actual tunnel section computed as the intersection between the surveyed terrain and the planned tunnel section. Thus, the topographer intervention is only required for the precise positioning of the TCS device at the beginning of each working stretch, which depends on the operational range of its components (limited to 25 meters in the current implementation). A prototype of the TCS system has been employed in a real construction site proving its benefits and advantages with respect to the traditional setting-out techniques.
@inbook{zuniga2019caip,
title = {Intrinsic Calibration of Depth Cameras for Mobile Robots using a Radial Laser Scanner},
author = {Zuñiga-Noël, David and Ruiz-Sarmiento, J. R. and Gonzalez-Jimenez, Javier},
doi = {10.1007/978-3-030-29888-3_54},
isbn = {978-3-030-29888-3},
year = {2019},
date = {2019-01-01},
volume = {11678},
pages = {659-671},
publisher = {Springer International Publishing},
series = {Computer Analysis of Images and Patterns. Lecture Notes in Computer Science},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
@inproceedings{antequera2019cvpr,
title = {Deep Single Image Camera Calibration with Radial Distortion},
author = {Lopez-Antequera, Manuel and Mari, Roger and Gargallo, Pau and Kuang, Yubin and Gonzalez-Jimenez, Javier and Haro, Gloria},
year = {2019},
date = {2019-01-01},
booktitle = {IEEE Int. Conference on Computer Vision and Pattern Recognition (CVPR)},
pages = {11817-11825},
address = {Long Beach, CA, USA},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{moreno2018appis,
title = {Analyzing Interference Between RGB-D Cameras for Human Motion Tracking},
author = {Moreno, Francisco-Angel and Ruiz-Sarmiento, J. R. and Monroy, Javier and Fernandez, Maïlys and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/papersrepo/2018/2018_moreno_APPIS Analyzing interference between RGBD cameras.pdf},
year = {2018},
date = {2018-01-01},
booktitle = {International Conference on Applications of Intelligent Systems (APPIS)},
address = {Las Palmas de Gran Canaria (Spain)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{briales2015minCalib,
title = {A minimal solution for the Calibration of a 2D Laser-Rangefinder and a Camera based on Scene Corners},
author = {Briales, Jesus and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/jbriales/publications/IROS15.pdf},
doi = {10.1109/IROS.2015.7353625},
year = {2015},
date = {2015-10-01},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015},
pages = {1891 - 1896},
address = {Hamburg, Germany},
organization = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{fernandezmoral2015b,
title = {Extrinsic Calibration of a Set of 2D Laser Rangefinders},
author = {Fernández-Moral, Eduardo and Arevalo-Espejo, Vicente and Gonzalez-Jimenez, Javier},
doi = {10.1109/ICRA.2015.7139475},
year = {2015},
date = {2015-05-01},
booktitle = {International Conference on Robotics and Automation},
address = {Seattle, Washington (USA)},
organization = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{gomez2015calib,
title = {Extrinsic Calibration of a 2D Laser-Rangefinder and a Camera based on Scene Corners},
author = {Gomez-Ojeda, Ruben and Briales, Jesus and Fernández-Moral, Eduardo and Gonzalez-Jimenez, Javier},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/rgomez/publications/icra15extrinsic.pdf},
doi = {10.1109/ICRA.2015.7139700},
year = {2015},
date = {2015-01-01},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
pages = {3611--3616},
address = {Seattle, USA},
organization = {IEEE},
abstract = {Robots are often equipped with 2D laser-
rangefinders (LRFs) and cameras since they complement well
to each other. In order to correctly combine measurements from
both sensors, it is required to know their relative pose, that is, to
solve their extrinsic calibration. In this paper we present a new
approach to such problem which relies on the observations of
an orthogonal trihedron which is profusely found as corners in
human-made scenarios. Thus, the method does not require any
specific pattern, which turns the calibration process fast and
simpler to perform. The estimated relative pose has proven
to be also very precise since it uses two different types of
constraints, line-to-plane and point-to-plane, as a result of a
richer configuration than previous proposals that relies on plane
or V-shaped patterns. Our approach is validated with synthetic
and real experiments, showing better performance than the
state-of-art methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Robots are often equipped with 2D laser-
rangefinders (LRFs) and cameras since they complement well
to each other. In order to correctly combine measurements from
both sensors, it is required to know their relative pose, that is, to
solve their extrinsic calibration. In this paper we present a new
approach to such problem which relies on the observations of
an orthogonal trihedron which is profusely found as corners in
human-made scenarios. Thus, the method does not require any
specific pattern, which turns the calibration process fast and
simpler to perform. The estimated relative pose has proven
to be also very precise since it uses two different types of
constraints, line-to-plane and point-to-plane, as a result of a
richer configuration than previous proposals that relies on plane
or V-shaped patterns. Our approach is validated with synthetic
and real experiments, showing better performance than the
state-of-art methods.
@inproceedings{fernandezmoral2014,
title = {Extrinsic calibration of a set of range cameras in 5 seconds without pattern},
author = {Fernández-Moral, Eduardo and Gonzalez-Jimenez, Javier and Rives, Patrick and Arevalo-Espejo, Vicente},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/varevalo/drafts/fernandez2014ecr.pdf},
year = {2014},
date = {2014-09-01},
booktitle = {International Conference on Intelligent Robots and Systems},
address = {Chicago, Illinois},
organization = {IEEE/RSJ},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@inproceedings{asensio2024smc,
title = {Sistema multi-cámara de estimación de pose sin marcadores para pHRI},
author = {Asensio-Huonder, Santiago and Fernández-Carmona, Manuel and Arevalo-Espejo, Vicente and Urdiales, Cristina and Gómez de Gabriel, Jesús Manuel},
doi = {10.17979/ja-cea.2024.45.10909},
year = {2024},
date = {2024-09-01},
journal = {Jornadas de Automática},
number = {45},
address = {Málaga},
organization = {Systems Engineering and Automation Dpt.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
@phdthesis{briales2021,
title = {Global Optimality via Tight Convex Relaxations for Pose Estimation in Geometric 3D Computer Vision},
author = {Briales, Jesus and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/22918},
year = {2021},
date = {2021-01-01},
school = {University of Málaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
@phdthesis{lopezantequera2019,
title = {Computer Vision Techniques for Calibration, Localization and Recognition},
author = {Lopez-Antequera, Manuel and Gonzalez-Jimenez, Javier},
url = {https://hdl.handle.net/10630/22879},
year = {2019},
date = {2019-01-01},
school = {Univerity of Malaga & University of Groningen},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
@phdthesis{fernandezmoral2014c,
title = {Contributions to Metric-Topological Localization and Mapping in Mobile Robotics},
author = {Fernández-Moral, Eduardo and Gonzalez-Jimenez, Javier and Arevalo-Espejo, Vicente},
url = {https://mapir.isa.uma.es/wp-content/uploads/publicaciones/legacy/varevalo/drafts/fernandez2014phd.pdf
http://hdl.handle.net/10630/9931},
year = {2014},
date = {2014-10-01},
school = {PhD. in Electrical Engineering, University of Malaga},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
