Chapter 12: Depth estimation

The most widely used datasets and benchmarks for two-frame and multi-view stereo are listed in Tables 12.1 and C.1. Among these, Middlebury stereo, KITTI, and ETH3D maintain active leaderboards tabulating the performance of two-frame stereo algorithms. For multi-view stereo, ETH3D and Tanks and Temples have leaderboards, and DTU is widely used and self-reported in papers.

• Many algorithms that train and test on the same dataset (e.g., KITTI) do not perform as well when tested on different datasets (Zendel et al. 2020). Song, Yang et al. (2021) discuss this issue and domain adaptation techniques that can reduce this problem.

KeystoneDepth has a large set of rectified historical image pairs, but without ground truth depth (Luo, Kong et al. 2020).

• For monocular depth inference, many algorithms train and test on the KITTI outdoor driving image sequences. The MiDaS system developed by Ranftl, Lasinger et al. (2020) federates a number of monocular depth inference datasets and also adds thousands of stereo image pairs from 3D movies for training, validation, and testing.

原书第 774 页

| --- | --- | --- | --- | --- | --- | --- | --- |

| ADE20K $ ^{{1}} $ | | | | | X | | |

| COCO $ ^{{2}} $ | | | | X | X | X | X |

| Cityscapes $ ^{{3}} $ | | | | | X | X | X |

| ETH3D $ ^{{4}} $ | X | | | | | | |

| HD1K $ ^{{5}} $ | | X | | | | | |

| KITTI $ ^{{6}} $ | X | X | X | | X | X | X |

| MVD $ ^{{7}} $ | | | | X | X | X | X |

| Middlebury $ ^{{8}} $ | X | X | | | | | |

| MPI Sintel $ ^{{9}} $ | | X | X | | | | |

| Objects365 $ ^{{10}} $ | | | | X | | | |

| OID $ ^{{11}} $ | | | | X | | X | |

| rabbitai $ ^{{12}} $ | | | X | | | | |

| ScanNet $ ^{{13}} $ | | | | | X | X | |

| VIPER $ ^{{14}} $ | | X | X | | X | X | X |

| WildDash $ ^{{15}} $ | | | | | X | X | X |

$ ^{1} $ http://sceneparsing.csail.mit.edu (Zhou, Zhao et al. 2019)

$ ^{2} $ http://cocodataset.org (Lin, Maire et al. 2014)

$ ^{3} $ https://www.cityscapes-dataset.com (Cordts, Omran et al. 2016)

$ ^{4} $ https://www.eth3d.net (Schöps, Schönberger et al. 2017)

$ ^{5} $ http://hci-benchmark.org (Kondermann, Nair et al. 2016)

$ ^{6} $ http://www.cvlibs.net/datasets/kitti (Menze and Geiger 2015)

$ ^{7} $ http://mapillary.com/dataset/vistas (Neuhold, Ollmann et al. 2017)

$ ^{8} $ http://vision.middlebury.edu (Scharstein, Hirschmüller et al. 2014)

$ ^{9} $ http://sintel.is.tue.mpg.de (Butler, Wulff et al. 2012)

$ ^{10} $ https://www.objects365.org (Shao, Li et al. 2019)

$ ^{11} $ https://storage.googleapis.com/openimages/web/index.html (Kuznetsova, Rom et al. 2020)

$ ^{12} $ https://rabbitai.de/benchmark (Schilling, Gutsche et al. 2020)

$ ^{13} $ http://kaldir.vc.in.tum.de/scannet_benchmark (Dai, Chang et al. 2017)

$ ^{14} $ https://playing-for-benchmarks.org (Richter, Hayder, and Koltun 2017)

$ ^{15} $ https://www.wilddash.cc (Zendel, Honauer et al. 2018)

Table C.1 The list of seven challenges (one per column) in the Robust Vision Challenge 2020 (http://www.robustvision.net) along with the datasets and benchmarks that are included in each challenge.