UMA Visual-Inertial Dataset

 

The UMA Visual Inertial Dataset is a collection of 32 sequences obtained in challenging conditions (changing light, low-terxtured scenes) with a handheld custom rig composed by a stereo camera, a stereo rig and a Inertial Measurement Unit.

 

 

The UMA-VI Dataset: Visual-Inertial Odometry in Low-textured and Dynamic Illumination Environments

-------------------------------------------------

Contact: David Zuñiga-Noël, Alberto Jaenal, Ruben Gomez-Ojeda, and Javier Gonzalez-Jimenez

Mails: This email address is being protected from spambots. You need JavaScript enabled to view it., This email address is being protected from spambots. You need JavaScript enabled to view it.

-------------------------------------------------

This paper presents a visual-inertial dataset gathered in indoor and outdoor scenarios with a handheld custom sensor rig, for over 80 min in total. The dataset contains hardware-synchronized data from a commercial stereo camera (Bumblebee®2), a custom stereo rig and an inertial measurement unit. The most distinctive feature of this dataset is the strong presence of low-textured environments and scenes with dynamic illumination, which are recurrent corner cases of visual odometry and SLAM methods. The dataset comprises 32 sequences and is provided with pseudo- ground truth poses at the beginning and the end of each of the sequences, thus allowing to measure the accumulated drift in each case. We also make available open source tools for evaluation purposes, as well as the intrinsic and extrinsic calibration parameters of all sensors in the rig.

-------------------------------------------------

Video

-------------------------------------------------

 

 

 

Calibration Sequences              Evaluation              Tools              License 

 

 

 

If you use this dataset, please cite it through (the paper is available here):     

 

@ARTICLE{
}

 

 

 

 

1. Summary

 

The dataset contains 32 sequences for the evaluation of VI motion estimation methods, totalling ∼80 min of data. The dataset covers challenging conditions (mainly illumination changes and low textured environments) in different degrees and a wide rage of scenarios (including corridors, parking, classrooms, halls, etc.) from two different buildings at the University of Malaga. In general, we provide at least two different sequences within the same scenario, with different illumination conditions or following different trajectories. All sequences were recorded with our VI sensor handheld, except a few that were recorded while mounted in a car.

 

 

 

 

2. Hardware 

We designed a custom VI sensor unit for data collection purposes. Our VI sensor consists of two stereo rigs and a three-axis inertial unit, as can be seen in the image below:

 

Sensor
Type
Quantity
Frequency (Hz)
Image size
Point Grey Bumblebee®2 (BB2-08S2C-25)
Stereo color camera (EquiDistant model)
1
12.5
1024x768x3
uEye UI-1226LE-M
Mono camera (RadTan model)
2 (Custom Stereo Rig)
25
752x480
Xsens MTi-28A53G35
Inertial Measurement Unit
1
250
 

 

 

 

 

 

 

3. Dataset Sequences

 

The dataset sequences have been classified depending on the type of challenge addressed. Each sequence is stored as a single .zip file and named following the format:

Name-BuildingNumber(-Reverse)_Date_Class

 All the secuence files contain (the format used is explained in the paper):

  • synchornized timestamps for camera and IMU
  • the ground truth poses for the start and the end of each sequence,  in order to allow a measure of the drift of the tested algorithm
  • the exposure time of each image

 

Category
Description
Number of sequences
Total time
Low-texture
indoor sequences showing environments lacking distinctive features or with repetitive textures
5
7.9 min
Indoor
sequences containing dark and light areas in indoor scenarios
5
8.4 min
Outdoor
sequences containing natural illumination changes present in outdoor environments
6
20.4 min
Indoor-Outdoor dynamic illumination
sequences containing the typical illumination changes of indoors/outdoors transitions
5
15 min
Indoor with illumination changes
indoor sequences with forced artificial lightning changes (spotlights, lights on and off)
5
10 min
Sun overexposure
particularly challenging sequences with a blinking effect in the uEye cameras caused by direct exposure to the sunlight, which saturates the imaging sensors and causes fast fluctuations in the exposure time
6
18.3 min
Challenging sequences
extra challenging sequences which were not counted on the dataset paper
1
1.3 min

 

 

 

 

4. Calibration Sequences 

 

The calibration parameters of our VI sensor can be downloaded here. We also provide the corresponding calibration sequence, allowing custom calibration methods to be used with our dataset.

The parameters of the AprilTag grid used in the calibration sequences are showen below. The file can also be downloaded here.

target_type: 'aprilgrid' #gridtype
tagCols: 6               #number of apriltags
tagRows: 6               #number of apriltags
tagSize: 0.0868          #size of apriltag, edge to edge [m]
tagSpacing: 0.2903       #ratio of space between tags to tagSize

 

 

Camera intrinsic calibration

 

We calibrated the intrinsic parameters (the projection parameters of each camera as well as the relative spatial transformation between the two cameras of each stereo setup) for each stereo rig independently. For that purpose, we recorded the calibration pattern (AprilTag grid) while slowly moving the VI sensor unit in front of it. The final calibration parameters were estimated using the calibration toolbox Kalibr. The next table shows three different sequences to calibrate each stereo pair.

 

Category
Description
Number of sequences
Total time
Bumblebee®2 intrinsic calibration sequences
sequences showing the calibration pattern while slowly moving the VI sensor unit in front of it
3
14.43 min
uEye intrinsic calibration sequences
sequences showing the calibration pattern while slowly moving the VI sensor unit in front of it
3
15.4 min

 

 

 

Camera photometric calibration

In order to allow direct VI approaches to be used with our dataset, we also provide calibration sequences for the sensor’s response function and lens vignetting mafor the uEye monochrome camera. To obtain the cameras response function, we recorded a static scene with different exposure times (ranging from 0.07 ms to 20 ms with the smallest steps allowed by the sensor). For the vignette calibration, we recorded a known marker (ArUco) on a white planar surface while moving the VI sensor in order to observe it from different angles. In the next table we provide two Response sequences and two Vignette sequences.

 

Category
Description
Number of sequences
Total time
uEye response calibration sequences
sequences showing static scene with different exposure times (ranging from 0.07 ms to 20 ms with the smallest steps allowed by the sensor)
2
10.94 min
uEye vignette calibration sequences
sequences showing the ArUco tag on a white planar surface while moving the VI sensor in order to observe it from different angles
2
3.42 min

 

The photometric camera calibration (A Photometrically Calibrated Benchmark For Monocular Visual Odometry) was performed only on the uEye camera (given that the Bumblebee parameters are continuously autoadjusted). To perform such calibration, we used the mono_dataset_code (edited by Alberto Jaenal). The vignette calibration results are shown in the next image (we calibrated the camera as a RadTan model):


Our results for the photometric calibration of each uEye camera are:


IMU Noise Calibration

The IMU intrinsics were obtained using the approach developed in The TUM VI Benchmark for Evaluating Visual-Inertial Odometry, in which the authors approximate the value of the deviation of white noise and bias by Allan deviations. The calibration sequence is avalaible for download as .bin and .npy files with the next formats:

 

Binary

# Unsigned long long timestamp, double gyro_x, gyro_y, gyro_z, accel_x, accel_y, accel_z (Little endian) 

NumPy

# timestamp, gyro_x, gyro_y, gyro_z, accel_x, accel_y, accel_z 

 

Category
Description
Number of sequences
Total time
IMU noise calibration sequence
sequences of our VI sensor resting capturing data for a long period
1
141.3 h

 

The calibration was performed with the IMU_calib_still tool. The results for the IMU intrinsic Noise Calibration are shown below:

 

imu0:
  T_i_b:
  - [1.0, 0.0, 0.0, 0.0]
  - [0.0, 1.0, 0.0, 0.0]
  - [0.0, 0.0, 1.0, 0.0]
  - [0.0, 0.0, 0.0, 1.0]

  #Accelerometers
  accelerometer_noise_density: 0.0007970737515806496 #Noise density (continuous-time)
  accelerometer_random_walk: 4.134509826460877e-06 #Bias random walk

  #Gyroscopes
  gyroscope_noise_density: 0.00046306229509090514 #Noise density (continuous-time)
  gyroscope_random_walk: 9.798339194666108e-07 #Bias random walk

  model: calibrated
  rostopic: /imu0/data #the IMU ROS topic
  time_offset: 0.0
  update_rate: 250.0  #Hz (for discretization of the values above)

 

Extrinsic calibration

 

We calibrated the extrinsics as well as the time-sinchronization offsets for each camera with respect to the IMU. For that purpose, we again recorded the calibration pattern (AprilTag grid) while moving the VI sensor unit in front of it, trying to excite the three axes of the IMU with rotation and translation movements. We used Kalibr to estimate the extrinsic parameters and time delays of the cameras.

 

Category
Description
Number of sequences
Total time
Bumblebee®2-Xsens extrinsic calibration sequences
sequences showing the calibration pattern while slowly exciting the three axes of the IMU with rotation and translation movements
1
2.83 min
uEye-Xsens extrinsic calibration sequences
sequences showing the calibration pattern while slowly exciting the three axes of the IMU with rotation and translation movements
1
3.13 min
Bumblebee®2-uEye-Xsens extrinsic calibration sequences
sequences showing the calibration pattern while slowly exciting the three axes of the IMU with rotation and translation movements
2
2.68 min

 

 The results for both the intrinsic calibration of each stero pair and its extrinsic calibration with the the IMU are provided below:

 

Bumblebee®2 - IMU
cam0:
  T_cam_imu:
  - [-0.9999193557683889, 0.0078087424182379074, 0.010015263430135759, 0.04560803849154623]
  - [-0.007982464004835765, -0.9998163145030406, -0.017424624012229308, 0.11460657726930963]
  - [0.009877359370849279, -0.017503165296644745, 0.9997980180898823, 0.00939792035366669]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [1]
  camera_model: pinhole
  distortion_coeffs: [-0.06983837053126551, 0.030679193251357234, -0.029318268716673087,
    0.008383563478792275]
  distortion_model: equidistant
  intrinsics: [545.7402000594014, 546.4624873938807, 516.7898455908171, 399.68834148863493]
  resolution: [1024, 768]
  rostopic: /cam0/image_raw
  timeshift_cam_imu: 0.009104475174114907
cam1:
  T_cam_imu:
  - [-0.9998724143729596, 0.005897711082467781, 0.014844931120653937, -0.07256189628225355]
  - [-0.0061818005321482745, -0.999797231887982, -0.019164562382438875, 0.11533023680091473]
  - [0.014728893990044001, -0.01925388566283171, 0.999706130604746, 0.008673101875104373]
  - [0.0, 0.0, 0.0, 1.0]
  T_cn_cnm1:
  - [0.99998651003287, 0.00182615045541887, 0.004862604939401894, -0.11842430675006582]
  - [-0.0018177951894695365, 0.9999968649306907, -0.0017221351400740366, 0.0008231093930239588]
  - [-0.0048657345726687215, 0.0017132726886604664, 0.999986694573365, -0.0006991291443189622]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [0]
  camera_model: pinhole
  distortion_coeffs: [-0.059680378064520846, 0.01009738651194745, -0.01025701371639368,
    0.002248862281197446]
  distortion_model: equidistant
  intrinsics: [543.6000821015224, 544.4238023037506, 524.280213672173, 381.4178892040695]
  resolution: [1024, 768]
  rostopic: /cam1/image_raw
  timeshift_cam_imu: 0.009130586492863666

 

 

uEye - IMU
cam0:
  T_cam_imu:
  - [-0.9996891001069981, -0.024321238097913023, 0.005493678607751144, 0.11425322792996404]
  - [0.024372821606514775, -0.9996575482196979, 0.009526376767667806, -0.013218431734117795]
  - [0.005260104010154635, 0.009657311466820284, 0.9999395319923278, -0.03159608818108305]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [1]
  camera_model: pinhole
  distortion_coeffs: [-0.11566634307511932, 0.08987699968528752, -0.0005582794020918663,
    -0.0018357664315637888]
  distortion_model: radtan
  intrinsics: [605.7455777299853, 607.9735414995575, 370.360573680134, 206.06348100221066]
  resolution: [752, 480]
  rostopic: /cam2/image_raw
  timeshift_cam_imu: 0.00534915928352496
cam1:
  T_cam_imu:
  - [-0.99928212777046, 0.035925730341547904, 0.012023768875391022, -0.13962875158190605]
  - [-0.03597274136607859, -0.9993458604025864, -0.003716607972848024, -0.016508639410168793]
  - [0.011882381796241693, -0.004146467851196176, 0.9999208047676615, -0.03160895767861787]
  - [0.0, 0.0, 0.0, 1.0]
  T_cn_cnm1:
  - [0.9981637475442355, -0.06015420959348745, 0.007113659862062496, -0.2542425622302439]
  - [0.06024646820793817, 0.9980914696176953, -0.01355659783506987, -0.010627124379391237]
  - [-0.006284596798540949, 0.013960277381725226, 0.9998828003813787, 0.0008859958970116961]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [0]
  camera_model: pinhole
  distortion_coeffs: [-0.12488375896406986, 0.10256449638718221, 0.00036679194033630247,
    0.00021222103272629827]
  distortion_model: radtan
  intrinsics: [604.6980984497027, 606.3456604476274, 365.754804193908, 235.0315549075409]
  resolution: [752, 480]
  rostopic: /cam3/image_raw
  timeshift_cam_imu: 0.005291164630008379

 

 

 

 

5. Evaluation 

In order to test each method's performance we developed evaluation tools, available at (repo). These tools test the the drift error and the alignment error, which are described in our paper, and have been developed in C++ (src/evaluate.cpp) and in Python3 (python/evaluate.py).

 

 

 

 

 

6. Tools 

 We also provide different tools in our (repo):

  • a rosbag sequence creator, developed in Python (python/rosbagCreator.py)
  • a TUM format to ASL format .csv sequence conversor, developed in C++ (src/tum2asl.cpp)
  • the aforedescribed IMU calibration toolbox (imu_calibration)

 

 

 

 

7. License 

All data in the UMA Visual Inertial Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY 4.0)

 

 

 

 

 

Data 

To obtain the rosbag files we originally recorded or another kind of data or information, please contact us

Calibration Parameters

MD5 checksum for all files


*: due to hardware limitations, there are some sequences in which some IMU messages were lost. However, the remain IMU messages are published in their proper time.

 

 

 

 

Low-texture

Sequence name

Info

Download

Video (5x)

class-csc1
Time:  105.12 s
Dim scene
Here

class-csc2 
Time:  80.56 s
Dim scene
Here

corridor-eng 
Time:  144.08 s
Here

parking-csc1 
Time:  72.32 s
Dim scene
Here

 parking-csc2
Time:  73.20 s
Dim scene
Here

 

 

 

 

Indoor

Sequence name

Info

Download

Video (5x)

class-eng 
Time:  71.12 s
Here

hall1-eng
Time:  96.64 s
Dim scene
Here

hall1-rev-eng 
Time:  65.92 s
Here

hall23-eng 
Time:  255.36 s
Dim scene
Here

third-floor-eng 
Time:  141.44 s
Here

 

 

 

 

Outdoor

Sequence name

Info

Download

Video (5x)

gattaca-csc1 
Time:  77.44 s
Here

 gattaca-csc2
Time:  237.36 s
Here

 long-corridor12-csc1
Time:  272.56 s
Here

long-corridor12-csc2 
Time:  262.96 s
2 IMU messages left*
Here

 long-corridor23-csc2
Time:  289.20 s
Here

parking-eng2 
Time:  86.16 s
Here

 

 

 

 

Indoor-Outdoor dynamic illumination

Sequence name

Info

Download

Video (5x)

lab-module-csc 
Time:  175.84 s
Here

lab-module-rev-csc 
Time:  181.76 s
Here

 long-walk-eng
Time:  267.2 s
Soft blinking
Here

 two-floors-csc1
Time:  132.40 s
Here

two-floors-csc2 
Time:  143.04 s
Here

 

 

 

 

Indoor with illumination changes

Sequence name

Info

Download

Video (5x)

conference-csc1
Time:  185.2 s
Here

conference-csc2 
Time:  94.08 s
Here

conference-csc3 
Time:  72.24 s
Here

third-floor-csc1 
Time:  76.40 s
Here

third-floor-csc2 
Time:  78.88 s
Here

 

 

 

 

Sun overexposure

Sequence name

Info

Download

Video (5x)

fantasy-csc1
Time:  147.92 s
Recorded on a car
Here

fantasy-csc2 
Time:  185.44 s
Recorded on a car
2 IMU messages left*
Here

hall1-two-floors-eng 
Time:  169.33 s
Here

long-corridor23-csc1 
Time:  264.64 s
Here

long-corridor23-rev-csc 
Time:  258.48 s
Here

parking-eng1 
Time:  72.64 s
Flash due to sun reflect
Here

 

 

 

 

Challenging sequences

Sequence name

Info

Download

Video (5x)

Elevator
Time:  80.96 s
Inside an elevator
 (IMU measures movement but image remains still)
Here

 

 

 

 

Intrinsic calibration

Bumblebee®2 calibration intrinsics

Sequence name

Info

Download

Video (5x)

Calib_cam-bumblebeeCLOSE 
Time:  139.65 s
Not IMU measures
Here

Calib_cam-bumblebeeMEDIUM 
Time:  358.41 s
Not IMU measures
Here

Calib_cam-bumblebeeCHANGE
Time:  367.55 s
Not IMU measures
Here

uEye calibration intrinsics

Sequence name

Info

Download

Video (5x)

Calib_cam-ueyeMEDIUM
Time:  341.80 s
Not IMU measures
Here

Calib_cam-ueyeFAR 
Time:  403.08 s
Not IMU measures
Here

Calib_cam-ueyeOUTDOOR
Time:  179.08 s
7 IMU messages left*
Here

 

 

 

 

Photometric calibration

Sequence name

Info

Download

Video (5x)

uEye response calibration

calib_response4 
Time:  276.65 s
Here

calib_response5 
Time:  379.63 s
Here

uEye vignette calibration

calib_vignette2 
Time:  76.41 s
Here

calib_vignette3 
Time:  128.64 s
Here

 

 

 

 

IMU noise calibration

Sequence name

Info

Download

UMAvi_imu_still

Time:  >141 h

IMU Frequency: 250 Hz 

Here: Binary

Here: NumPy

 

 

 

 

Extrinsic calibration

Bumblebee®2 - Xsens extrinsic calibration

Sequence name

Info

Download

Video (5x)

Calib_imu-bumblebeeENG2 
Time:  169.99 s
Not uEye measures
Here

uEye - Xsens extrinsic calibration

Sequence name

Info

Download

Video (5x)

Calib_imu-ueyeENG2 
Time:  187.83 s
Not Bumblebee®2 measures
Here

Bumblebee®2 - uEye - Xsens extrinsic calibration

Sequence name

Info

Download

Video (5x)

Calib_imu-camBRIDGE
Time:  81.83 s
3 messages left*
Here

Calib_imu-camOUTDOOR 
Time:  78.95 s
Here