Chapter 11: Structure from motion and SLAM

• OpenCV implements a number of widely used camera calibration and pose estimation algorithm in the calib3d module, as does OpenGV (https://laurentkneip.github.io/opengv) (Kneip and Furgale 2014) and OpenMVG (https://github.com/openMVG/openMVG) (Moulon, Monasse et al. 2016).

• You can find an experimental comparison of a number of RANSAC variants at https://opencv.org/evaluating-opencvs-new-ransacs/.

• A large number of open-source bundle adjustment algorithms designed to handle unordered photo collections have been developed over the years, including:

• SBA: sparse bundle adjustment (https://www.ics.forth.gr/~lourakis/sba) (Lourakis and Argyros 2009).

– Simple sparse bundle adjustment (SSBA) (https://github.com/chzach/SSBA).

• Bundler, structure from motion for unordered image collections (https://phototour.cs.washington.edu/bundler) (Snavely, Seitz, and Szeliski 2006).

• The Ceres Solver for bundle adjustment and general non-linear least squares (http://ceres-solver.org).

– MCBA (Multicore Bundle Adjustment) (https://grail.cs.washington.edu/projects/mcba) (Wu, Agarwal et al. 2011).

– Visual SfM (http://ccwu.me/vsfm), which wraps a GUI around several reconstruction algorithms (Wu, Agarwal et al. 2011; Wu 2013).

– MVE (https://www.gcc.tu-darmstadt.de/home/proj/mve), a complete SfM pipeline with densification, meshing, and texturing (Fuhrmann, Langguth et al. 2015).

– The Theia global structure from motion library (http://www.theia-sfm.org) (Sweeney, Hollerer, and Turk 2015).

• OpenMVG (Open Multiple View Geometry) https://github.com/openMVG/openMVG (Moulon, Monasse et al. 2016).

• COLMAP (https://github.com/colmap/colmap), which includes both a large-scale structure from motion system (Schönberger and Frahm 2016) and a multi-view stereo pipeline (Schönberger, Zheng et al. 2016).

– Square Root Bundle Adjustment (https://vision.in.tum.de/research/vslam/rootba) (Demmel, Sommer et al. 2021).

Among these, COLMAP appears to be the most often used today in other research projects, e.g., for image-based rendering systems.

• Popular open-source packages for Simultaneous Localization and Mapping (SLAM) and Visual Odometry (VO or VIO) include

– LSD-SLAM (large-scale direct SLAM) (Engel, Schöps, and Cremers 2014).

• ORB-SLAM (Mur-Artal, Montiel, and Tardos 2015) and ORB-SLAM2 (Mur-Artal and Tardós 2017),

– SVO (semi-direct visual odometry) (Forster, Zhang et al. 2017),

– GTSAM (Dellaert and Kaess 2017; Dellaert 2021),

– DSO (direct sparse odometry) (Engel, Koltun, and Cremers 2018).

– BAD SLAM (bundle adjusted direct RGB-D SLAM) (Schöps, Sattler, and Pollefeys 2019a), and

– GSLAM (a general SLAM framework and benchmark) (Zhao, Xu et al. 2019).

There are also highly-optimized SLAM/VIO libraries available on mobile platforms, such as iOS (ARKit), Android (ARCore), and Facebook (Spark AR Studio), designed for easy integration into mobile augmented reality applications.