Benchmarking and Comparing Popular Visual SLAM Algorithms

  • Amey Kasar


This paper contains the performance analysis and
benchmarking of two popular visual SLAM Algorithms: RGBDSLAM
and RTABMap. The dataset used for the analysis is the
TUM RGBD Dataset from the Computer Vision Group at TUM.
The dataset selected has a large set of image sequences from a
Microsoft Kinect RGB-D sensor with highly accurate and time
synchronized ground truth poses from a motion capture system.
The test sequences selected depict a variety of problems and
camera motions faced by SLAM algorithms for the purpose of
testing the robustness of SLAM algorithms in different situations.
The evaluation metrics used for the comparison are Absolute
Trajectory Error (ATE) and Relative Pose Error (RPE). The
analysis involves comparing the Root Mean Square Error
(RMSE) of the two metrics and the processing time for each
algorithm. This paper serves as an important aid in the selection
of SLAM algorithm for different scenes and camera motions. The
analysis helps to realize the limitations of both SLAM methods.
This paper also points out some underlying flaws in the used
evaluation metrics

Keywords: Simultaneous Localization And Mapping, Benchmark, RGBD SLAM, RTABMap


[1] R. C. Smith and P. Cheeseman, “On the representation and estimation of
spatial uncertainty,” The international journal of Robotics Research, vol.
5, no. 4, pp. 56–68, 1986.
[2] G. Dissanayake, H. Durrant-Whyte, and T. Bailey, “A computationally
efficient solution to the simultaneous localisation and map building
(slam) problem,” in Robotics and Automation, 2000. Proceedings.ICRA’00. IEEE International Conference on, vol. 2, pp. 1009–1014,
IEEE, 2000.
[3] M. Montemerlo and S. Thrun, “Simultaneous localization and mapping
with unknown data association using fastslam,” in ICRA, pp. 1985–
1991, 2003.K. Elissa, “Title of paper if known,” unpublished.
[4] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A
benchmark for the evaluation of rgb-d slam systems,” in Proc. of the
International Conference on Intelligent Robot Systems (IROS), Oct.
[5] R. Kümmerle, B. Steder, C. Dornhege, M. Ruhnke, G. Grisetti, C.
Stachniss, and A. Kleiner, “On measuring the accuracy of slam
algorithms,” Autonomous Robots, vol. 27, no. 4, p. 387, 2009
[6] F. Steinbrücker, J. Sturm, and D. Cremers, “Real-time visual odometry
from dense rgb-d images,” in Computer Vision Workshops (ICCV
Workshops), 2011 IEEE International Conference on, pp. 719–722,
IEEE, 2011
[7] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous
driving? the kitti vision benchmark suite,” in Computer Vision and
Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 3354–
3361, IEEE, 2012.
[8] ] F. Endres, J. Hess, N. Engelhard, J. Sturm, D. Cremers, and W.
Burgard, “An evaluation of the rgb-d slam system,” in Robotics and
Automation (ICRA), 2012 IEEE International Conference on, pp. 1691–
1696, IEEE, 2012
[9] M. Labbé and F. Michaud, “Memory management for real-time
appearance-based loop closure detection,” in Intelligent Robots and
Systems (IROS), 2011 IEEE/RSJ International Conference on, pp.
1271– 1276, IEEE, 2011
[10] TUM, “Rgb-d slam dataset and benchmark.” datasets/rgbd-dataset, 2012. Accessed:
[11] G. Grisetti, R. Kümmerle, C. Stachniss, U. Frese, and C. Hertzberg,
“Hierarchical optimization on manifolds for online 2d and 3d mapping,”
in Robotics and Automation (ICRA), 2010 IEEE International
Conference on, pp. 273–278, IEEE, 2010
[12] B. K. Horn, “Closed-form solution of absolute orientation using unit
quaternions,” JOSA A, vol. 4, no. 4, pp. 629–642, 1987
[13] A. Kelly, “Linearized error propagation in odometry,” The International
Journal of Robotics Research, vol. 23, no. 2, pp. 179–218, 2004
[14] A. Kasar, “Git repository for the results and scripts.”
ameykasar/benchmarking-slam, 2018. Accessed: 2018-11-22
0 Views | 0 Downloads
How to Cite
Kasar, A. (2019). Benchmarking and Comparing Popular Visual SLAM Algorithms. Asian Journal For Convergence In Technology (AJCT). Retrieved from