Chapter 11: Structure from motion and SLAM

• The Benchmark for 6DOF Object Pose (BOP) developed by Hodaň, Michel et al. (2018) has results from the recent challenge and workshop at https://bop.felk.cvut.cz/challenges/bop-challenge-2020 and http://cmp.felk.cvut.cz/sixd/workshop_2020.

• The Long-Term Visual Localization Benchmark, https://www.visuallocalization.net, includes datasets such as Aachen Day-Night (Sattler, Maddern et al. 2018) and InLoc (Taira, Okutomi et al. 2018) along with an associated set of challenges and workshop held at ECCV 2020.

• The 1DSfM collection of landmark images created by Wilson and Snavely (2014) (https://www.cs.cornell.edu/projects/1dsfm), which is an extension of the Photo Tourism dataset created by Snavely, Seitz, and Szeliski (2008a), is widely used to test large-scale structure from motion algorithms. The poses provided with this dataset, which were obtained using the software in Wilson and Snavely (2014), are generally considered as “ground truth” when testing more efficient algorithms, although they have never been geo-registered. The ETH3D, https://www.eth3d.net (Schöps, Schönberger et al. 2017) and Tanks and Temples, https://www.tanksandtemples.org (Knapitsch, Park et al. 2017) datasets are also occasionally used.

• Some widely used benchmarks for SLAM systems include a benchmark for RGB-D SLAM systems (Sturm, Engelhard et al. 2012), the KITTI Visual Odometry / SLAM benchmark (Geiger, Lenz et al. 2013), the synthetic ICL-NUIM dataset (Handa, Whelan et al. 2014), the TUM monoVO dataset (Engel, Usenko, and Cremers 2016), the EuRoC MAV dataset (Burri, Nikolic et al. 2016), the ETH3D SLAM benchmark (Schöps, Sattler, and Pollefeys 2019a), and the GSLAM general SLAM framework and benchmark (Zhao, Xu et al. 2019). Many of these are surveyed and categorized in the paper by Ye, Zhao, and Vela (2019), which was presented at the ICRA 2019 Workshop on Dataset Generation and Benchmarking of SLAM algorithms for Robotics and VR/AR, https://sites.google.com/view/icra-2019-workshop/home.